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SUMMARY 


As  information  technologies  are  becoming  more  central  to  weapons  systems,  the  burden  is 
shifting  from  conventional  human  factors  requirements,  to  cognitive  requirements.  This  report 
explores  the  feasibility  of  different  methodologies  for  capturing  and  understanding  the  cognitive 
requirements  in  envisioned  worlds.  The  goal  of  the  project  was  to  identify  methodologies  that 
could  be  leveraged  in  a  tool  to  help  Program  Managers  and  Directors  of  Engineering  incorporate 
cognitive  requirements  early  in  the  design  cycle.  Three  methodologies  are  tested:  Cognitive  Task 
Analysis  ( CTA ),  scenarios,  and  Team  Integrated  Design  Environment  (TIDE).  CTA  involves 
knowledge  elicitation  interviews  and  analysis  to  understand  the  cognitive  and  decision-making 
aspects  of  tasks.  Scenarios  are  a  form  of  knowledge  representation  in  which  the  cognitive 
requirements  are  described  in  the  context  of  a  mission.  TIDE  is  a  tool  for  determining  optimal 
team  configurations. 

These  three  methodologies  were  tested  in  the  context  of  Airborne  Laser  (ABL)  Battle 
Management  Command,  Control,  Communications,  Computers,  and  Intelligence  (BMC4I).  The 
Airborne  Laser  is  a  program  under  development  in  which  the  crew  is  under  pressure  to  make 
rapid  decisions  about  whether  or  not  to  fire  a  chemical  laser  at  enemy  missiles.  Information 
about  cognitive  requirements  was  collected  via  observation  and  CTA  interviews.  Various 
formats  (including  a  scenario)  were  used  to  illustrate  these  cognitive  requirements,  and  optimal 
team  configurations  were  explored  via  TIDE.  Some  of  the  major  findings  from  the  project  are 
described  below: 

CTA  was  useful  in  flagging  the  cognitively  challenging  aspects  of  the  ABL  mission  and 
organizing  the  tasks  involved  into  ten  high-level  functions.  These  cognitive  challenges  could  be 
used  to  specify  human  computer  interaction  recommendations. 

Team  dynamics  and  potential  breaking  points  were  illustrated  with  a  scenario  of  a  potential  ABL 
mission.  TIDE  was  able  to  more  evenly  distribute  workload  across  the  crew  and  demonstrate  the 
impact  of  reducing  crew  size. 

Interviews  with  perspective  users  indicated  that  input  about  crew  decision  making  would  be 
valuable  in  developing  the  concept  of  operation  as  well  as  design,  function  allocation,  staffing, 
and  training. 
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These  findings  were  used  to  develop  initial  ideas  for  a  tool  to  conceptualize  the  dynamics  and 
tradeoffs  involved  in  cognitive  tasks.  Scenarios  form  the  basis  for  this  tool,  called  Cognitive 
Requirements  for  Individuals  and  Teams:  Evaluations,  Recommendations,  Inspection,  and 
Analysis  or  CRITERIA.  The  intent  is  that  CRITERIA  would  capture  cognitive  criteria  for  tasks 
in  complex  systems  and  illustrate  the  implications  through  the  use  of  simulation. 
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INTRODUCTION 


This  Phase  I  Small  Business  Innovative  Research  (SBIR)  report  addresses  the  technology  that  would 
provide  a  means  for  Program  Managers  to  incorporate  cognitive  requirements  into  the  design  of  new 
systems.  As  information  technologies  become  more  central  to  weapons  systems,  the  burden  is  shifting 
from  conventional  human  factors  requirements  to  the  cognitive  requirements  of  operating  systems 
designed  around  data  flow  and  information  management.  System  designers  need  tools  that  enable  them  to 
anticipate  the  cognitive  demands  placed  on  the  operators  of  these  systems.  Moreover,  cognitive 
requirements  needs  to  be  addressed  early  in  the  concept  development  stage  of  design,  while  a  full  range  of 
options  is  still  available.  If  the  cognitive  engineering  is  delayed  until  late  in  the  design  cycle,  the  degrees 
of  freedom  for  improving  the  system  will  usually  be  too  few.  As  constraints  add  up  and  the  design  moves 
towards  closure  —  it  becomes  increasingly  costly  to  make  changes  to  accommodate  the  needs  of  operators 
and  users. 

The  discipline  of  cognitive  engineering  was  developed  in  part  to  represent  human  requirements  in  order  to 
improve  design.  Unfortunately,  cognitive  engineering  has  not  yet  had  the  impact  that  is  needed.  The  key 
goal  in  this  Phase  I  effort  was  to  make  the  transition  from  cognitive  engineering  strategies  applied  in 
research  and  development  efforts  to  more  general  and  practical  strategies  that  would  provide  information 
useful  for  Program  Managers  and  designers. 

Barriers  to  Cognitive  Engineering  Application  in  New  Programs 

Barriers  to  Achieving  Program  Emphasis  for  Cognitive  Engineering 

Currently,  cognitive  engineering  methods  are  not  applied  until  systems  have  achieved  a  reasonable  state 
of  maturity  --  when  it  is  already  too  late  to  significantly  influence  the  design.  The  types  of  data  relevant 
to  cognitive  engineering  that  Program  Managers  need  are  difficult  to  collect  early  in  the  system  design 
process.  Further,  a  Program  Manager  is  typically  facing  a  variety  of  hard,  near-term  challenges  that 
demand  his  or  her  attention.  For  example,  the  system  may  require  hardware  or  software  breakthroughs 
that  must  be  achieved  in  order  to  keep  the  project  viable  ~  lack  of  these  breakthroughs  would  then  render 
cognitive  engineering  results  moot  in  any  event.  In  the  case  of  the  Airborne  Laser  (ABL)  program,  the 
challenge  is  to  design  a  laser  that  is  sufficiently  small  and  light  to  be  carried  on  a  Boeing  747-400F,  yet 
sufficiently  powerful  to  function  as  a  weapon.  Meanwhile,  the  ABL  Program  Manager  has  to  also  resolve 
safety  concerns,  such  as  the  stability  of  laser  operations,  the  ability  to  deconflict  the  use  of  the  laser  so  as 
to  prevent  fratricide,  and  the  ability  to  handle  hazardous  materials  involved  in  laser  operations.  Collateral 
issues  such  as  the  size  and  weight  of  fire  suppressants  also  emerge  periodically  to  demand  the  attention  of 
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the  Program  Manager.  The  technical  and  logistical  challenges  faced  by  the  Program  Manager  are 
considerable.  In  such  a  context,  concerns  about  cognitive  requirements  take  on  less  immediate 
importance.  When  all  this  is  coupled  with  often-present  uncontrolled  variables  -  such  as  the  ABL’s 
situation  wherein  the  analogue  system’s  (the  Airborne  Warning  and  Control  System  or  AWACS)  crew 
task  loading  is  not  as  demanding  —  it  becomes  very  difficult  to  gain  sufficient  attention  or  emphasis  on 
cognitive  requirements. 

Barriers  to  Applying  Cognitive  Engineering  to  Team  Sizing  and  Allocation  of  Function 
Cognitive  engineering  methods  have  primarily  been  applied  to  individual  workstations.  However  when  a 
Program  Manager  needs  to  be  concerned  with  team  functions  at  an  early  stage  of  design,  the  team 
composition  and  the  requirements  for  team  coordination  are  of  equal  or  greater  concern  than  are  the 
requirements  of  individual  operators.  Cognitive  engineering  studies  need  to  extend  the  current  research  to 
address  team  coordination,  rather  than  continuing  to  emphasize  design  for  individuals.  The  current 
methods  for  improving  human-computer  interfaces  focus  on  aiding  the  individual  operator.  Methods  for 
Cognitive  Task  Analysis  (CTA)  almost  exclusively  support  individuals.  It  is  often  assumed  that  a  team  is 
a  collection  of  individuals,  and  if  we  improve  the  performance  of  all  the  individual  members,  then  the 
team  performance  should  also  benefit.  However,  this  bottom-up  approach  ignores  the  emergent 
properties  of  teams  (e.g.,  communication  needs,  information  management,  coordination  issues, 
requirements  for  shared  situation  awareness),  and  its  underlying  assumption  is  not  always  valid. 

Barriers  to  recommending  team  size.  The  field  of  human  factors  has  fewer  methods  for  recommending 
team  size  than  it  has  for  specifying  individual  cognitive  requirements.  A  bottom-up  strategy  is  unlikely  to 
address  the  emergent  properties  of  teams.  For  example,  Klinger  and  Klein  (1999)  have  described  a 
program  where  staffing  of  the  emergency  response  organization  of  a  nuclear  power  plant  was  excessive, 
and  performance  gains  were  achieved  by  dramatically  cutting  the  staffing.  Workload  actually  went  down 
as  fewer  staff  members  were  incorporated  in  the  team.  If  we  are  to  support  teams,  we  need  to  capture 
these  emergent  processes.  We  need  to  specify  the  requirement  for  shared  situation  awareness.  We  need 
to  determine  how  skilled  teams  are  able  to  maintain  self-awareness.  We  need  to  develop  methods  for 
helping  teams  establish  useful  mental  models  of  the  roles  and  functions  of  each  of  the  members.  We  need 
to  identify  the  ways  that  teams  pool  their  understanding  to  detect  problems  and  make  inferences. 

Barriers  to  recommending  team  member  requirements.  Designing  for  teams  is  difficult  because  we  do 
not  have  good  methods  for  representing  team  performance  requirements.  Existing  methods  for  describing 
teamwork  often  translate  into  massive  wiring  diagrams  of  interactions  between  each  of  the  team 
members.  This  is  another  example  of  a  bottom-up  approach.  The  assumption  is  that  by  specifying  all  the 
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interactions,  the  insights  must  be  someplace  inside  the  proliferation  of  connection  lines.  In  actuality, 
these  types  of  comprehensive  diagrams  are  not  very  intuitive,  and  are  not  very  useful  for  guiding  the 
design  process.  The  complexity  of  team  interaction  is  conveyed,  but  without  the  meaning  of  those 
interactions. 

Decision-Centered  Design  Approach  to  Supporting  Team-Design  Decisions 

This  Phase  I  effort  explored  an  innovative  strategy  —  a  Decision-Centered  Design  (DCD)  approach  to 
develop  a  tool  to  address  the  cognitive  requirements  of  teams  in  a  new,  envisioned  system.  A  decision- 
centered  design  approach  relies  on  extensive  front-end  analysis  to  understand  the  problem.  It  also  means 
working  iteratively  with  potential  users  to  obtain  their  feedback  and  ensure  that  their  needs  are  being  met. 
The  DCD  process  is  illustrated  in  Figure  1.  Cognitive  Task  Analysis  interviews  are  conducted  with 
decision  makers  in  order  to  understand  their  decision  requirements  or  cognitive  demands.  These  decision 
requirements  are  then  used  to  recommend  changes  to  design,  staffing,  or  training.  This  process  is 
repeated  at  several  points  in  the  project  to  ensure  the  decision  makers’  needs  are  being  met. 


Too  often,  decision  support  systems  are  not  designed  around  the  decision-making  aspects  of  the  task.  As 
a  result,  the  systems  fail  to  provide  the  necessary  information,  fail  to  provide  it  in  a  useful  form,  or  --  as  is 
often  the  case  -  make  it  more  difficult  to  access  essential  information.  In  the  field  of  human  factors  this 
is  known  as  clumsy  automation  (Woods,  Johanessen,  Cook,  &  Sarter,  1994),  because  the  good  intentions 
of  the  designers  result  in  worse  performance  rather  than  improved  performance.  The  decision-centered 
design  approach,  on  the  other  hand,  begins  with  a  CTA  study  to  identify  the  decision  requirements  of  the 
task,  i.e.,  the  key  judgments  and  decisions,  the  reasons  why  they  are  difficult,  the  types  of  errors  that  are 
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found,  and  the  patterns  and  strategies  used  by  experienced  personnel.  What  is  striking  about  this 
approach  is  that  it  does  not  begin  with  decompositions  into  basic  procedure-oriented,  perceptual-motor 
tasks  or  determinations  of  information  flow.  Instead,  it  fits  within  user-centered  design  approaches  (e.g., 
Landauer,  1995)  by  focusing  on  the  decision  requirements  for  performing  the  task  well,  and  uses  these  to 
design  the  architecture  of  an  information-management  system. 

Klein  Associates  has  successfully  applied  aspects  of  DCD  in  previous  design  projects  (Klinger  &  Gomes, 
1993;  Miller  &  Lim,  1993)  with  high  degrees  of  success.  Klinger,  Andriole,  Militello,  Adelman,  Klein, 
and  Gomes  (1993)  report  a  careful  evaluation  that  determined  that  performance  of  an  AW  ACS  Weapons 
Director  was  significantly  improved  by  an  interface  design  based  on  decision  requirements.  Miller  and 
Lim  (1993)  designed  a  decision  support  system  for  Air  Force  weaponeers.  It  was  enthusiastically 
received  by  the  operational  users,  and  its  sponsors  moved  rapidly  to  system  development. 

However,  we  have  also  been  learning  that  having  feasible  and  useful  methods  is  not  a  guarantee  that  they 
will  be  adopted.  We  have  learned  that  it  is  essential  to  influence  design  from  the  beginning,  when  the 
statement  of  need  is  issued  by  the  design  requirements  analysts,  and  the  program  office  conceptualizes  a 
strategy  for  satisfying  the  need.  As  the  system  is  taking  shape  —  as  the  weapons  system  is  being  defined 
--  it  is  critical  to  articulate  the  cognitive  requirements  of  the  individuals  and  the  crew  who  will  be 
operating  and  supporting  it. 

In  this  project,  we  have  begun  to  develop  a  decision-centered  design  methodology  for  use  during  the 
conceptual  design  of  systems,  so  that  the  program  designers  have  a  tool  for  considering  cognitive 
requirements.  Such  a  tool  will  enable  them  to  generate  more  effective  designs,  to  identify  important 
questions  that  could  trigger  research  activities  early  on,  to  set  cognitive  criteria  for  the  contractors  who 
will  be  engaged  in  the  detailed  system  design,  and  to  specify  acceptance  criteria  to  be  used  during  test  and 
evaluation. 

The  use  of  a  DCD  tool  would  allow  the  using  command  to  look  downstream  and  safeguard  the  operators 
who  will  eventually  inherit  the  system.  It  is  too  easy  for  the  operators’ needs  to  be  overlooked  during 
system  development  because  the  focus  and  milestones  are  typically  on  hardware  and  software  delivery, 
and  the  compromises  that  reduce  usability  are  often  invisible  to  the  operational  community.  By  placing 
cognitive  requirements  in  the  mix  of  design  criteria  from  the  beginning,  we  hope  to  afford  protection  to 
the  operators  by  making  their  needs  a  viable  part  of  the  design  process  from  the  beginning.  Failure  to  do 
so  will  lead  to  systems  that  create  barriers  to  the  operators’  understanding  of  situations,  and  lead  to  human 
error. 
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Overview  of  the  Phase  I SBIR  Approach 

The  overall  goal  of  this  project  was  to  help  Program  Managers  address  cognitive  requirements  of  teams 
early  in  the  design  cycle.  In  order  to  develop  a  software  tool  to  accomplish  this  we  conducted  two  major 
series  of  tasks.  One  series  of  tasks  involved  testing  the  feasibility  of  different  methods  to  determine  how 
well  they  would  support  a  Program  Manager.  These  methods  included:  Cognitive  Task  Analysis,  Team 
Integrated  Design  Environment  (TIDE)  software,  and  scenarios.  The  domain  of  ABL  was  used  as  a 
testbed  for  these  methods  and  tools.  Rather  than  attempt  a  highly-detailed  study  (which  would  take  far 
too  long,  require  far  too  much  data,  and  would  require  types  of  data  that  would  not  be  available  until  late 
in  the  design  cycle),  Klein  Associates  explored  what  could  be  accomplished  with  initial  data  as  a 
framework  for  exploring  the  design  tradeoffs  involving  the  crew  members.  The  results  of  each  method 
and  feasibility  for  use  in  a  Phase  II  software  tool  are  discussed  in  the  Results  of  Phase  I  section  of  this 
report. 

Two  series  of  tasks  (illustrated  in  Figure  2)  were  conducted  in  parallel  in  order  to  accomplish  our  overall 
goal  to  develop  system  concepts  for  a  tool  to  support  Program  Managers.  CTA  interviews  were 
conducted  with  Program  Managers  in  order  to  understand  the  challenges  of  program  management  and 
how  we  could  support  those  challenges. 
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Figure  2.  Two  major  tasks  in  Phase  I. 


System 
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There  were  several  innovations  to  this  project.  Program  Managers  and  directors  of  engineering  were 
targeted  as  the  user,  rather  than  design  engineers  who  are  responsible  for  system  development.  This  is  a 
critical  shift.  While  the  findings  from  a  preliminary  DCD  would  be  relevant  to  design  engineers,  we 
believe  that  the  greatest  impact  can  be  realized  by  incorporating  cognitive  requirements  into  the  initial 
design  concepts.  The  second  innovation  was  to  use  a  DCD  approach  to  focus  on  decisions  rather  than 
procedures.  The  third  innovation  was  to  focus  on  the  team  rather  than  individuals. 
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The  advantage  of  the  DCD  approach  is  that  it  offers  Program  Managers  and  design-requirements  analysts 
a  means  for  taking  cognitive  requirements  into  account  early  in  the  design  cycle,  and  for  anticipating  the 
consequences  of  decisions  such  as  crew  size,  qualifications  level,  configuration,  and  training 
requirements. 

Rather  than  trying  to  provide  recommendations,  we  have  concluded  from  the  Phase  I  effort  that  it  is  more 
feasible  and  useful  to  present  the  findings  in  the  form  of  decision  scenarios.  Program  Managers  can  use 
these  scenarios  to  understand  the  dynamics  of  how  the  crew  will  have  to  make  key  judgments  and 
decisions  and  will  have  to  coordinate  during  critical  events.  Instead  of  trying  to  represent  workload  as  an 
amorphous  concept,  we  are  trying  to  capture  the  specific  types  of  coordination  patterns  that  may  be  found 
during  high  workload  periods.  Following  the  work  of  Schwartz  (1991),  we  are  using  decision  scenarios 
as  a  means  for  learning  about  tradeoffs,  instead  of  attempting  a  premature  set  of  recommendations. 

METHODS 

Three  methods  seemed  promising  in  their  ability  to  capture  and  represent  cognitive  requirements  of 
teams.  During  Phase  I,  we  tested  each  of  these  methods  in  the  domain  of  ABL  to  determine  which 
methods,  or  aspects  of  the  methods,  were  most  promising  for  inclusion  in  the  Program  Manager  tool. 

One  requirement  for  selection  is  that  the  methods  work  in  the  early  stages  of  system  development  when 
less  information  about  the  system  and  the  users  is  available.  The  three  methods  tested  were: 

•  Cognitive  Task  Analysis  (CTA)  for  capturing  and  representing  aspects  of  teams  and  cognition  in 
an  envisioned  world. 

•  TIDE  for  providing  insight  into  the  optimal  configuration  of  teams  and  the  assignment  of  roles 
and  functions. 

•  Scenarios  as  a  tool  for  representing  the  critical  decisions  and  aspects  of  team  coordination  in 
context. 

In  addition,  we  conducted  CTA  interviews  with  Program  Managers  to  understand  their  challenges  and 
how  they  could  best  be  supported. 


Airborne  Laser  Background 

The  domain  of  ABL  was  chosen  to  test  feasibility  of  methods  for  several  reasons.  The  program  is 
relatively  early  in  the  development  cycle  and  therefore  could  benefit  from  insight  into  the  cognitive 
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elements  of  the  task.  The  ABL  program  was  also  dealing  with  issues  related  to  staffing  —  the  allocation 
of  roles  and  functions  among  crew  members  and  the  identification  of  appropriate  Air  Force  specialty 
codes  (AFSCs)  to  fill  each  position.  These  issues  made  ABL  a  good  candidate  to  test  feasibility  of  CTA 
and  TIDE  for  envisioned  worlds. 

ABL  is  a  weapon  system  designed  to  shoot  down  theater  ballistic  missiles  during  the  boost  phase  in  the 
first  line  of  defense.  Once  a  missile  launch  is  detected,  two  illumination  lasers  are  used  to  acquire  the 
target  and  provide  information  regarding  air  turbulence.  The  beam  control  system  computes  information 
about  target  acquisition  and  compensates  for  atmospheric  distortion.  A  high  energy  chemical  laser  then 
strikes  the  missile  so  early  that  the  missile  and  its  warhead  do  not  leave  the  country  that  launched  it.  All 
this  is  accomplished  from  a  modified  Boeing  747  flying  behind  the  engagement  zone.  One  of  the 
interesting  aspects  of  ABL  is  that  --  because  of  the  speed  of  the  events  involved  --  the  laser  will  normally 
shoot  down  the  missile  automatically.  It  is  the  crew’s  job  to  determine  whether  or  not  to  stop  the  shot. 
This  is  true  command  by  negation. 

In  researching  the  ABL,  it  is  easy  to  find  information  about  the  physical  systems  on  board  —  the  different 
lasers  that  will  be  used  and  the  physics  involved.  News  stories  are  full  of  details  about  the  systems  but 
often  fail  to  mention  the  crew  who  will  be  operating,  monitoring,  and  keeping  those  systems  functional. 
As  we  began  to  get  involved  in  the  ABL  program,  we  heard  more  concerns  about  the  crew  members  and 
efforts  to  understand  crew  requirements.  In  fact,  it  was  crew  composition  and  crew  dynamics  that  was 
one  of  the  unknown  factors.  Our  goal  in  the  Phase  I  ABL  analysis  was  to  understand  the  critical  decisions 
the  crew  will  face  as  a  whole,  how  difficult  those  decisions  were,  and  why  they  were  difficult. 
Understanding  this  information  will  benefit  the  design,  staffing,  and  training  in  the  ABL  program. 

Cognitive  Task  Analysis  of  Airborne  Laser 

CTA  provides  a  set  of  tools  for  eliciting  general  domain  knowledge  and  specific  knowledge  pertaining  to 
the  decision  requirements  for  the  critical  decisions.  CTA  comprises  both  knowledge  elicitation 
(interviews  and  observations  with  subject  matter  experts)  and  knowledge  representation  (analysis  and 
meaningful  representation  of  the  data). 

The  CTA  of  ABL  allowed  us  to  go  beyond  the  procedural  textbook  knowledge  and  the  behavioral  aspects 
of  a  task  that  are  traditionally  elicited  and  represented  by  a  behavioral  task  analysis.  The  tools  we  used 
allowed  us  to  understand  some  of  the  cognitive  aspects  of  the  ABL  mission  --  in  particular  the  judgment, 
decision-making,  and  problem-solving  skills  that  are  so  critical  in  the  time-pressured,  uncertain,  and  ever- 
changing  air  combat  environment. 
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In  this  effort,  a  full  CTA  was  not  feasible  for  several  reasons.  This  is  an  envisioned  problem  so  ABL 
experts  do  not  exist  and  we  had  to  adjust  our  methods  accordingly.  We  did  talk  to  subject  matter  experts 
of  analogue  systems  such  as  AW  ACS.  Often  in  new  systems  there  is  a  shortage  of  time  to  interview  and 
study  experts.  Therefore,  we  conducted  an  abbreviated  CTA  in  which  we  tried  to  identify  the  cognitively 
challenging  tasks  and  decisions  for  the  individuals  and  the  crew. 

In  preparation  for  the  CTA,  we  met  with  an  ABL  subject  matter  expert  for  half  a  day  to  learn  about 
program  status,  current  design  approach  and  constraints,  technical  aspects  of  ABL,  and  what  they  would 
want  out  of  our  cognitive  engineering  projects.  The  second  knowledge  elicitation  was  conducted  at 
Kirtland  Air  Force  Base  during  the  Joint  Expeditionary  Force  Exercise  1999  (JEFX-99).  We  were  able  to 
observe  a  subset  of  the  Battle  Management  Crew  as  they  participated  in  the  simulation.  We  were  able  to 
observe  how  the  crew  reacted  to  routine  tasks  such  as  monitoring  the  situation  and  refueling,  as  well  as 
their  reaction  to  non-routine  events  such  as  missile  launches  and  enemy  attacks.  We  were  able  to  discern 
information  about  critical  decisions  and  judgments  of  different  crew  members  and  the  crew  as  a  whole. 
Other  observations  focused  on  coordination  issues,  information  needs,  and  interactions  with  the  displays. 

In  addition  to  observations,  we  conducted  informal  interviews  with  the  crew  members.  We  were  able  to 
ask  questions  during  down-times  —  both  during  the  simulations  and  at  the  end  of  the  day.  Our  questions 
were  aimed  at  understanding  the  difficult  decisions,  judgments,  and  coordination  issues.  Sample 
questions  included: 

•  What  is  difficult  about  the  ABL  mission,  and  what  is  difficult  about  each  particular  job  position? 

•  What  aspects  of  your  training  prepared  you  for  this  assignment? 

•  What  are  the  challenging  decisions  that  you  would  not  trust  to  someone  with  less  experience? 

•  What  experience  or  knowledge  is  needed  to  make  those  decisions? 

•  What  information  do  you  need  to  make  that  decision? 

•  What  are  the  difficult  teamwork  issues? 

•  What  are  you  seeing  and  hearing,  and  how  is  that  affecting  your  actions? 

•  At  different  points  in  the  mission  --  who  do  you  want  sitting  next  to  you  and  why? 
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To  facilitate  analysis,  detailed  notes  of  the  interviews  and  observations  were  generated.  We  then  had  to 
determine  how  to  make  sense  of  the  large  amount  of  context-rich  data  collected.  We  chose  to  identify  the 
cognitively  challenging  tasks  and  information  about  why  the  tasks  are  challenging.  In  tailoring  CTA 
methods  to  envisioned  worlds,  it  is  important  to  identify  potential  cognitive  demands  early  so  they  can  be 
supported  by  the  design,  staffing,  or  training.  This  is  the  type  of  information  we  identified  during  Phase  I. 

In  order  to  accomplish  this,  Klein  Associates  generated  a  list  of  the  tasks,  decisions,  and  functions 
involved  in  the  ABL  mission.  We  limited  our  analysis  to  the  “engagement”  phase  of  the  mission  -  the 
events  between  takeoff  and  landing,  excluding  pre-mission  planning  and  post-mission  debrief.  The  tasks, 
decisions,  and  functions  were  at  a  mid-level  of  granularity.  They  were  at  a  higher  level  than  tasks  such  as 
button-pushing  and  at  a  more  detailed  level  than  “maintain  situation  awareness.”  Examples  of  this  mid 
level  include  “determine  best  orbit  placement,”  and  “monitor  enemy  air  tracks.” 

The  tasks,  functions,  and  decisions  were  then  sorted  into  ten  major  categories  or  functions.  The  items 
were  sorted  according  to  what  overall  function  the  task  played  in  the  ABL  mission.  For  example, 
“monitor  enemy  tracks”  could  be  part  of  maintain  plan  flow  as  well  as  self-protection.  For  each  task,  we 
used  our  knowledge  of  the  domain  and  information  from  interviews  and  observations  to  determine  how 
cognitively  challenging  the  task  was,  why  the  task  was  cognitively  challenging,  and  other  cognitive 
information  associated  with  the  task.  We  applied  ratings  of  high,  medium,  and  low  to  reflect  how 
cognitively  challenging  we  judged  a  task  to  be.  Low  indicated  that  the  task  was  not  very  cognitively 
challenging  and  that  little  or  no  additional  CTA  data  were  needed  on  the  subject.  Medium  indicated  that 
there  was  a  fair  amount  of  cognitive  complexity  involved  and  the  topic  would  benefit  from  additional 
CTA  probing.  A  rating  of  high  indicated  a  large  degree  of  cognitive  complexity  and  expertise  was 
involved  and  that  additional  CTA  would  be  beneficial  to  further  understand  the  cognitive  nature  of  these 
tasks.  For  each  task  listed  in  Table  1  and  the  ten  tables  in  Appendix  A  (under  the  column  entitled 
“Decision/function”),  a  description  of  what  we  found  that  made  the  task  cognitively  challenging  is 
included  (under  the  column  entitled  “Why  challenging”). 

The  heart  of  the  ABL  analysis  was  the  development  of  a  scenario  to  bring  to  life  the  critical  decisions, 
judgments,  and  coordination  involved  in  the  ABL  mission.  In  developing  the  scenario,  a  list  of  several 
key  events  and  accompanying  decisions  was  developed.  These  events  were  then  organized  sequentially 
in  a  timeline  and  the  details  were  fleshed  out  in  the  story.  The  scenario  was  then  annotated  to  highlight 
the  cognitive  and  team  aspects  of  the  tasks. 
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In  addition  to  the  above  analyses,  we  made  several  sweeps  through  the  data  to  pull  out  the  major  findings. 
These  were  findings  that  could  have  a  major  impact  on  design,  staffing,  or  the  development  of  training 
requirements. 


TIDE  Analysis 

Analysis  was  also  conducted  using  Team  Integrated  Design  Environment  (TIDE)  to  analyze  workload 
and  determine  the  optimal  team  configuration.  The  TIDE  organizational  design  methodology  was 
mission-driven.  That  is,  the  model  used  information  about  the  tasks  required  to  accomplish  a  mission  and 
the  resources  available  to  accomplish  those  tasks,  and  used  algorithms  to  optimally  allocate  these  tasks 
and  resources  to  team  members  to  create  an  organizational  structure.  To  capture  the  tactical  and 
operational  elements  in  a  scenario,  the  research  team  relied  on  input  from  the  CTA. 


Mission 
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STAGE  II 

Figure  3.  TIDE  three-part  allocation  model. 

The  TIDE  approach  was  based  on  a  three-part  allocation  model,  presented  in  Figure  3,  that  considers: 

1)  mission  tasks,  2)  system  resources,  and  3)  human  decision  makers  (the  ABL  crew).  The  organizational 
design  process  is,  in  simplest  terms,  an  algorithm-based  allocation  between  these  three  parts.  In  TIDE, 
organizational  performance  is  assumed  to  be  a  function  of  a  variety  of  design  parameters  —  including 
individual  workload,  distribution  of  responsibility,  communication  between  decision  makers, 
coordination  of  resources,  information  processing  efficiency,  and  information  transfer  efficiency.  To 
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apply  the  TIDE  methodology,  we  needed  to  know  the  sequence  in  which  tasks  were  performed,  the 
resources  that  were  used  to  perform  each  task,  and  the  interdependencies  among  tasks.  Given  this 
information  for  a  specific  mission  scenario,  our  modeling  techniques  suggested  ways  that  tasks  should  be 
grouped  together  (i.e.,  handled  by  the  same  person  or  the  same  group  of  people)  in  that  scenario  in  order 
to  both  satisfy  organizational  constraints  and  optimize  performance  according  to  different  possible  criteria 
(e.g.,  equalizing  workload  across  people). 

Task  Decomposition. 

Using  the  CTA  data,  the  ABL  mission  was  divided  into  three  overall  mission  phases  or  functions: 
Monitoring,  Self  Defense,  and  Missile  Elimination.  Within  each  of  these  functions,  a  series  of  mission 
tasks  was  specified.  For  example,  “monitor  screen  and  alerts  for  missile  launch”  is  a  task  within  the 
monitoring  function.  Each  of  these  tasks  was  classified  into  Action,  Decision,  Information,  and  Outcome 
categories.  Using  these  components  a  task  diagram  was  developed  for  each  mission  phase  to  understand 
the  functional  and  temporal  relationships  between  the  tasks.  An  example  of  a  task  dependency  diagram  is 
presented  in  Figure  4. 

In  addition  to  defining  the  inter-task  relationship,  the  relative  time  to  complete  each  task  (in  minutes)  and 
the  relative  workload  (on  a  scale  of  1-5)  of  each  task  were  estimated.  Workload  for  each  action  and 
outcome  task  was  an  estimate  of  the  relative  effort.  For  the  decision  tasks,  workload  was  an  estimate  of 
cognitive  effort.  For  the  information  tasks,  workload  was  defined  as  processing  effort. 

Interviews  with  Program  Managers 

Since  our  eventual  goal  under  Phase  I  was  to  build  a  support  tool  for  Program  Managers,  we  conducted 
interviews  to  understand  our  users’  needs.  We  interviewed  a  total  of  four  Program  Managers  from  both 
the  military  and  the  commercial  sector.  Each  interview  lasted  approximately  one  hour.  These  were  high- 
level  Program  Managers  in  charge  of  the  development  of  new  systems.  The  purpose  of  the  interviews 
was  to  understand  the  challenging  aspects  of  the  Program  Manager’s  job,  especially  as  it  relates  to 
incorporating  cognitive  requirements  into  the  system  design  requirements.  In  addition  to  understanding 
challenges,  we  collected  information  about  their  information  needs:  which  tools  and  methodologies 
worked  and  which  didn’t,  what  type  of  information  was  useful  at  different  stages  of  development,  and 
what  information  they  wished  they  had  known  sooner.  Understanding  the  challenges,  decisions,  and 
information  needs  of  Program  Managers  will  better  enable  us  to  support  them. 
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Figure  4.  Task  dependency  diagram. 


RESULTS  OF  PHASE  I 
Overview 

Included  in  our  key  findings  was  that  we  could  identify  individual  and  team  requirements,  and  that  there 
were  surprises  between  the  tough  decisions  we  identified  and  the  ones  we  were  briefed  about.  For 
example,  we  were  told  that  one  of  the  toughest  decisions  the  crew  would  have  to  make  was  determining 
whether  to  take  the  shot  in  a  short  window  of  time.  We  observed  that  the  system  decides  whether  to  take 
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the  shot,  and  the  human  monitors  the  situation  and  may  decide  to  prevent  the  shot.  We  were  then  told  that 
the  tough  decision  was  deconfliction  -  determining  whether  any  aircraft  were  in  the  path  of  the  laser. 
However  in  observing  the  simulation  and  talking  with  operators,  this  did  not  appear  to  be  very  cognitively 
challenging.  This  is  because  the  system  automatically  deconflicts  in  automatic  mode.  In  most  cases 
when  the  system  is  in  automatic,  the  operator  will  not  have  to  worry  about  deconfliction  (as  we  observed 
in  the  simulation).  The  only  time  the  operator  will  have  to  perform  deconfliction  is  in  manual  mode.  The 
challenges  in  this  case  are  that  the  operator  needs  to  think  three-dimensionally  to  determine  if  there  is 
anything  in  the  path  of  the  laser  and  that  s/he  needs  to  do  this  under  extreme  time  pressure. 


The  major  tasks  and  decisions  were  organized  into  ten  high-level  functions  (see  Figure  5).  These  are 
functions  that  the  crew  as  a  whole  needs  to  accomplish.  They  are  not  broken  out  by  position.  This  is 
because,  ideally,  in  an  envisioned  world  this  analysis  would  be  done  before  the  number  or  roles  of 
personnel  have  been  decided.  Proximity  to  other  functions  indicates  how  related  one  function  is  to  the 
other  functions.  These  are  not  discrete  categories;  there  is  overlap  of  tasks  and  decisions  among  the 
categories.  Each  of  the  functions  is  briefly  described  below. 

Self  Protection.  There  is  considerable  overlap  between  this  function  and  Maintain  Plan  Flow.  Both 
functions  involve  monitoring  enemy  tracks,  Combat  Air  Patrol  (CAP)  location,  and  status  of  sensors  and 
communications.  Self  Protection  also  involves  critical  functions  of  gauging  threat  to  ABL  and 
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determining  when  and  where  to  retrograde.  Gauging  threat  to  ABL  requires  identification  of  the  type  of 
threat.  Often,  it  is  the  missile  the  aircraft  can  carry,  rather  than  the  type  of  aircraft  that  is  the  determining 
factor.  The  crew  must  think  ahead  and  anticipate  enemy  actions. 

Maintain  Plan  Flow.  This  function  encompasses  everything  the  crew  does  to  obtain  and  interpret 
information.  Some  of  the  critical  tasks  are:  detecting  problems  and  inconsistencies,  monitoring  track 
number  changes,  applying  commander’s  intent,  and  orchestrating  priorities.  One  of  the  challenges  that 
comes  up  again  and  again  in  this  category  is  resource  allocation.  Resource  allocation  is  relevant  because 
there  is  a  limit  to  how  much  information  a  person  can  process  at  a  time,  there  are  a  limited  number  of 
people  to  accomplish  a  variety  of  tasks,  and  many  of  the  tasks  are  critical  and  need  to  be  accomplished 
immediately. 

•  Aircraft  Placement.  Aircraft  placement  affects  the  ABL’s  readiness  to  shoot.  Launch  sites  and 
times  must  be  anticipated  in  advance  so  the  ABL  is  in  a  position  to  shoot.  This  involves 
discerning  trends  of  missile  launches.  There  are  several  aspects  of  aircraft  placement  that  must  be 
considered;  these  include:  placement  of  the  orbit  so  the  ABL  covers  known  launch  sites  but  is 
not  at  risk,  type  of  orbit  (circle,  figure  eight,  etc.)  so  the  launch  sites  are  covered  the  maximum 
amount  of  time,  and  placement  within  the  orbit.  During  the  exercise,  we  heard  the  Mission  Crew 
Commander  (MCC)  ask  several  times  how  long  it  would  be  until  the  aircraft  made  the  turn  [and 
would  be  pointing  in  the  right  direction  to  shoot  down  enemy  missiles].  This  function  is  made 
more  difficult  because  it  will  require  team  coordination  between  the  several  members  of  the  battle 
management  crew  and  the  flight  crew  (the  crews  are  not  co-located). 

•  Determine  System  Status.  This  function  is  located  near  the  center  of  the  diagram  to  illustrate  its 
relevance  to  many  of  the  other  functions.  In  order  to  accomplish  many  of  the  other  functions, 
such  as  Maintain  Plan  Flow  and  Determine  Readiness  to  Shoot,  the  crew  needs  to  know  whether 
the  systems  are  functioning  correctly  and  whether  they  can  trust  the  data  on  their  displays.  The 
crew  must  be  adept  at  noticing  when  something  is  wrong  with  the  systems.  This  is  not  a  trivial 
task,  as  it  often  involves  “seeing  what  is  not  there”  —  in  other  words,  what  data  were  expected  and 
what  data  are  missing? 

•  Track  Detection  and  Identification.  This  function  is  important  and  is  applicable  to  Maintaining 
Plan  Flow,  Self  Defense,  and  Determining  Whether  to  Stop  the  Shot.  Operators  must  locate 
tracks  by  number,  use  standardized  voice  tell  formats,  and  accept  or  reject  automatic 
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identification  recommendations.  Accepting  or  rejecting  recommendations  requires  an  experience 
base  to  detect  errors  and  detect  subtle  cues  about  the  tracks. 

•  Readiness  to  Shoot.  Readiness  to  shoot  refers  to  whether  the  ABL  is  capable  of  shooting  a 
missile  at  any  given  time.  This  is  something  that  must  be  continually  assessed  to  ensure  that  the 
amount  of  time  during  which  the  ABL  is  capable  of  taking  a  shot  is  maximized.  Some  factors 
involved  are:  whether  the  aircraft  is  turned  the  correct  way,  the  weapon  status  (whether  the  laser 
is  ready  to  shoot),  and  the  status  of  the  link. 

•  Decide  Whether  to  Stop  the  Shot.  This  may  be  a  straightforward  decision  if  the  system  is  in 
automatic  mode.  In  the  automatic  mode,  the  system  performs  deconfliction  with  aircraft  and 
prioritization  among  missiles.  The  main  difficulty  in  automatic  mode  would  be  deciding  to 
reprioritize  based  on  new  information  that  has  not  yet  been  entered  into  the  system’s  prioritization 
scheme.  Deciding  whether  to  stop  the  shot  when  the  system  is  in  manual  mode  is  much  more 
cognitively  challenging  because  the  human  must  perform  deconfliction  and  prioritization. 
Deconfliction  in  manual  mode  involves  forming  a  three-dimensional  picture  of  the  world  to  see  if 
there  is  anything  in  the  projected  path  of  the  laser.  In  order  to  perform  prioritization  in  manual 
mode,  the  operator  must  have  an  understanding  of  the  prioritization  scheme  and  be  able  to  apply 
it.  Another  difficulty  in  deciding  whether  to  stop  the  shot  has  to  do  with  resource  allocation  -  in 
terms  of  how  many  shots  the  laser  has  left.  This  is  not  a  preset  number,  it  must  be  estimated 
based  on  range  from  the  missile,  when  the  missile  can  be  engaged,  and  how  long  the  missile  must 
be  engaged.  The  operator  may  decide  to  stop  the  laser  from  firing  at  a  lower  priority  target  in 
anticipation  of  a  higher  priority  target. 

•  Adjustments  During  Engagement.  This  involves  all  the  tasks  and  decisions  that  may  need  to  be 
performed  once  the  decision  has  been  made  to  allow  the  shot.  Some  examples  are:  acquire  target 
(in  manual  mode),  determine  if  the  missile  is  at  a  good  altitude  to  shoot,  determine  why  the 
system  will  not  engage  (in  automatic  mode),  perform  deconfliction,  and  determine  how  long 
before  the  missile  can  be  engaged.  Estimating  the  time  until  missile  engagement  is  dependent  on 
the  angle  of  the  missile,  the  altitude  of  the  missile,  and  the  ABL’s  placement  in  its  orbit.  This 
information  is  important  because  it  will  help  the  decision  maker  determine  if  there  is  enough  time 
to  engage  the  missile  or  if  the  shot  will  be  lost.  None  of  the  tasks  are  trivial  on  their  own. 
Deconfliction  involves  special  discrimination,  and  there  is  only  a  short  window  of  time  to  take  the 
shot.  These  tasks  are  made  even  more  challenging  by  the  fact  that  they  all  must  be  accomplished 
in  a  short  period  of  time. 
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•  Deal  with  Emergency  Situations.  These  are  functions  that  would  not  normally  occur  during  an 
ABL  mission  that  went  as  planned.  However,  these  are  events  and  decisions  that  the  crew  must 
be  ready  to  handle.  Some  examples  are:  deal  with  chemical  leak,  determine  whether  to  perform 
an  emergency  landing,  and  recognize  jamming.  In  many  of  these  functions,  the  danger  to  the 
crew  must  be  gauged  and  the  implications  of  actions  must  be  understood. 

•  Keep  the  System  Working.  These  are  all  the  tasks  involved  in  the  upkeep  of  the  different  systems 
onboard  the  ABL.  Many  of  the  tasks  are  routine  and  procedural,  such  as  initialize  computers  and 
switch  configurations  between  users.  Other  tasks  and  decisions  are  more  difficult,  including 
prioritizing  which  system  to  fix  first  if  multiple  systems  simultaneously  fail  and  determining  how 
to  work  with  degraded  systems. 

Another  major  finding  had  to  do  with  crew  composition.  One  of  the  assumptions  is  that  AW  ACS  is  a 
good  analogue  to  ABL.  However,  there  are  significant  differences  between  the  MCC  roles  on  AW  ACS 
and  Airborne  Laser.  Airborne  Laser  is  a  single  mission  platform  while  AW  ACS  can  have  multiple, 
simultaneous  missions.  One  implication  of  this  is  that  the  AWACS  MCC  may  be  overqualified  for  the 
job. 


Cognitively  Challenging  Aspects  of  Airborne  Laser  Missions 

This  section  describes  the  cognitive  challenges  of  the  ABL  mission.  Each  of  the  high-level  functions  in 
Figure  5  contained  many  subtasks  and  decisions.  The  sheer  number  of  subtasks  and  decisions  in  each 
function  make  most  of  the  high-level  functions  appear  cognitively  challenging.  Therefore,  in  order  to 
decipher  which  major  aspects  of  the  ABL  mission  were  cognitively  challenging,  we  examined  the 
individual  tasks  and  decisions.  For  each  task,  we  rated  how  cognitively  challenging  it  was  (low,  medium, 
high),  described  why  it  was  cognitively  challenging,  and  provided  additional  CTA  data  or  questions  to  be 
answered.  A  sample  of  the  type  of  information  presented  is  extracted  from  Table  A-2  and  presented  in 
Table  1 .  The  complete  set  of  tables  for  each  of  the  ten  major  functions  can  be  found  in  Appendix  A. 

The  CTA  approach  was  valuable  in  identifying  the  cognitive  elements  involved  in  the  ABL  mission  as 
well  as  team  coordination  aspects.  The  CTA  was  useful  in  determining  which  aspects  of  the  task  are 
cognitively  challenging  and  why  they  are  challenging.  We  believe  this  information  would  be  useful 
because  it  identifies  the  high  drivers  or  leverage  points  in  terms  of  cognitive  and  team  elements.  This 
information  could  then  be  used  in  determining  function  allocation  and  staffing  requirements.  We  also 
believe  that  this  type  of  information  would  be  available  early  in  the  design  process  (although  at  a  lower 
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level  of  resolution  or  fidelity).  All  these  factors  suggest  that  CTA  will  be  a  valuable  tool  in  eliciting  team 
cognitive  requirements  in  an  envisioned  world. 


Table  1.  Some  Cognitive  Challenges  involved  in  the  ABL  Mission 


Decision/function 

- 

CC 

Why  challenging 

Additional  CTA  data 

2.1  Monitor  enemy  tracks 

Med. 

There  are  multiple  tracks  that 
can  disappear  and  reappear. 

The  enemy  will  often  be  trying 
to  use  deception  so  operators 
need  to  maintain  enemy 
perspective  to  make  sense  of 
what  they  see  on  the  screen. 

Tracks  can  carry  weapons  that 
can  be  a  threat  from  farther 
away  than  the  original  track 
itself. 

Need  to  project  ahead  and 
understand  implications  of 
enemy  actions. 

- 

2.2  Filter  and  sort  information 

High 

Need  to  make  meaning  and 
draw  conclusions  from  the 
information  (synthesize). 

If  information  is  ambiguous, 
missing,  or  incorrect,  it  can 
make  filtering  and  sorting 
difficult. 

If  too  much  information  is 
coming  in,  it  can  be  difficult  to 
identify  which  information  is 
relevant. 

Operators  need  to  filter  the 
information  that  is 
displayed.  All  data  blocks 
are  not  the  same;  it  depends 
on  the  track.  For  example, 
altitude  is  not  necessary  for 
ground  and  surface  tracks. 
Modes  and  codes  are  not 
necessary  for  all  tracks. 

2.3  Determine  whether  sensors 
and  communications  are 
working 

High 

Must  notice  inconsistency  in 
data.  In  other  words,  you  need 
to  see  what  is  not  there. 

Indicators  may  not  reflect 
exactly  what  the  system  is 
actually  doing.  It  is  easy  to 
assume  that  information  is 
being  sent  and  received  if  there 
are  no  indicators  to  let  you 
know  something  is  not  working. 

CC  stands  for  Cognitively  Challenging 
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Table  1.  Some  Co* 

jnitive  Challenges  involved  in  the  ABL  Mission  (continued) 

Decision/function 

cc 

Why  challenging 

Additional  CTA  data 

2.4  Determine  where  strikes 
are  taking  place 

Low 

Information  is  available  in  the 

Air  Tasking  Order  (ATO)  and 
on  the  screen. 

A  maneuvering  surface  to 
air  missile  (SAM)  suggests 
a  strike  package  is  taking 
place. 

2.5  Detect  problems  and 
inconsistencies  in  track  data 

High 

This  is  dependent  upon  the 
number  of  sensors  you  have  out 
there,  the  sensitivity  of  the 
sensors,  and  the  location  of  the 
sensors. 

Inconsistencies  may  not  stand 
out,  or  they  may  occur 
frequently  enough  that  the 
important  ones  do  not  stand  out 
more  than  the  unimportant 
ones. 

Need  experience  base  to 
compare  situation  and 
recognize  anomalies. 

Display  aid  -  a  tool  could 
be  built  to  track  certain 
aspects  during  critical 
salient  events. 

It  is  difficult  to  maintain 
situation  awareness  and  not 
narrow  your  focus  during  a 
critical  event. 

A  display  aid  could  show 
relevant  information 
without  distracting  the 
operator  from  the  focus. 

2.6  Know  and  apply 
commander’s  intent 

High 

Involves  applying  commander’s 
intent  to  the  current  situation 
and  determining  if  your  actions 
are  consistent. 

This  can  be  difficult  if  the 
commander’s  intent  is  poorly 
articulated  or  focused  at  the 
wrong  level  of  granularity. 

2.7  Switch  configuration 
between  users 

Low 

Need  to  determine  the  level  of 
information  you  need  to  know 
(big  picture  versus  specifics). 

This  needs  to  be  done 
quickly  without  re-logging 
in  as  a  new  user. 

If  the  display  and  its 
accompanying  switch 
actions  to  do  this  task  are 
complex  and  deeply 
embedded  in  menus,  it  can 
make  the  task  much  more 
difficult. 

2.8  Perform  spatial  and  audio 
discrimination  between  voice 
communications 

Low 

Need  to  determine  who  is 
speaking  (source  of 
information)  at  the  same  time 
you  are  listening  for  relevant 

1  information. 
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Table  1.  Some  Cognitive  Challenges  involved  in  the  ABL  Mission  (continued) 


Decision/function 

CC 

_ g  .  — - 

Why  challenging 

Additional  CTA  data 

2.9  Monitor  location  of  high 
value  assets  (HVAs) 

Med. 

Can  be  difficult  if,  for  security 
reasons,  the  locations  of  the 
HVAs  are  not  widely 
disseminated. 

Some  individuals  on  Airborne 
Laser  may  not  have  high 
enough  clearance  to  be  entitled 
to  this  information. 

Display  aid  -  have  the  HVA 
stand  out  by  placing  a  circle 
around  them  or  using  a 
different  color. 

2. 10  Monitor  track  number 
changes 

Med. 

Need  to  integrate  information 
from  multiple  sources. 

One  difficulty  with  track 
number  changes  is  that 
operators  often  type  in  the  track 
number  of  interest  (once  they 
‘learn’  it)  and  a  change  in 
numbers  can  have  them 
inadvertently  looking  at  the 
wrong  track.  Operators  may 
need  to  unlearn  the  old  number 
as  well  as  leam  the  new 
number. 

Tracks  may  be  picked  up 
by  multiple  sensors, 
especially  in  joint 
operations  (e.g.,  by  a  U.S. 
AW  ACS,  a  NATO 

AW  ACS,  an  E-2C,  an 

AEGIS  cruiser).  Given  the 
possibility  for  multiple 
sensors  to  acquire  the  track, 
there  is  a  need  for  all  of 
these  linked  data  sources  to 
be  “correlated.”  There  is  a 
chance  that  tracks  can  be 
given  multiple  track 
numbers  until  the  system 
correlates.  As  a  result, 
there  could  be  situations 
where  a  track  number  could 
be  changed. 

Scenario  of  an  Airborne  Laser  Mission 

This  section  demonstrates  the  use  of  a  scenario  in  representing  cognitive  and  team  issues.  The  purpose  is 
to  explore  the  usefulness  of  a  scenario  in  representing  these  issues,  not  to  predict  what  actually  might 
occur  in  an  ABL  mission.  One  of  the  major  challenges  in  data  analysis  and  representation  is  in  capturing 
the  dynamics  and  bringing  to  life  the  context  in  which  a  new  system  will  operate.  We  believe  that 
scenario  tools  fill  this  void  and  will  enable  Program  Managers  to  better  communicate  their  intent.  The 
following  story  is  an  example  of  the  scenario  approach.  The  story  is  a  fictitious  demonstration  of  a 
portion  of  an  Airborne  Laser  mission,  told  from  the  perspective  of  the  battle  management  crew.  The  story 
illustrates  a  sampling  of  the  cognitive  and  decision-making  elements  of  an  Airborne  Laser  mission.  The 
story’s  narrative  is  on  the  left.  Margin  notes  are  included  on  the  right  to  indicate  the  decisions,  cues, 
factors,  or  difficulties  in  making  decisions.  The  margin  notes  are  the  key  to  communicating  the  specific 
cognitive  aspects  to  be  supported. 


19 


SCENE:  A  troubled  corner  of  the  Middle  East  in  the  near  future.  The  Airborne  Laser  entered  the 
situation  one  week  ago.  Their  mission  is  to  shoot  down  enemy  missiles  before  the  missiles  reach  friendly 
territory.  The  battle  management  current  crew  consists  of  Major  Doug  Warren  (Mission  Crew 
Commander),  Major  Hal  Krasneiwski  (Air  Surveillance  Officer),  Lieutenant  Tom  Collins  (Weapons 
System  Officer),  and  Sergeant  Mike  Ramsey  (Technician). 

ACTION  DECISION/COGNITIVE 

REQUIREMENTS 


“Something’s  wrong,”  said  Lieutenant  Collins  as  he 
studied  his  computer  screen.  Collins  focused  on  the 
displays  in  front  of  him.  One  provided  a  bird’s  eye 
perspective  of  the  earth  beneath  him  along  with  tracks 
indicating  friendly  and  enemy  assets.  The  other  display 
was  like  a  slice  through  the  air  picture  in  which  he  could 
see  the  altitude  of  all  the  aircraft  in  relation  to  the 
Airborne  Laser.  As  Weapons  System  Officer,  it  was  his 
job  to  make  sense  of  the  displays  and  stay  aware  of 
friendly  and  enemy  aircraft  locations. 

Something  doesn  ’t  seem  right  he  thought.  He  peered  at 
the  two  perspectives,  integrating  them  into  a 
3-dimensional  picture  in  his  mind.  There,  that’s  it  -  the 
tracks  on  the  left  hand  of  the  display  are  not  matching 
up.  Collins  noticed  that  some  of  the  tracks  on  the  bird’s 
eye  view  display  were  not  showing  up  on  the  altitude 
display.  He  turned  to  the  technician  across  the  small 
space.  “Hey,  Sergeant  Ramsey,  take  a  look  at  these 
screens.  The  displays  don’t  seem  to  be  matching  up.” 

What  now?  thought  Ramsey.  He  was  in  the  middle  of 
running  a  diagnostic  on  the  laser.  What  was  supposed  to 
be  a  routine  task  had  turned  out  to  be  more  problematic 
than  expected.  The  diagnostic  indicated  an  error  with  the 
pressure  sensor  in  one  of  the  chemical  chambers. 

Ramsey  walked  over  to  the  Weapons  System  Officer  to 
see  what  the  problem  was.  As  soon  as  Collins  pointed 
out  the  mismatch,  Ramsey  realized  what  must  be  wrong. 
There  must  be  something  wrong  with  the  data  link  from 
AWACS. 


Decision:  Determine  if  the  systems  are 
working  correctly 

Cue:  Must  see  what  is  not  there,  know  the 
system  well  enough  to  recognize  when 
something  abnormal  occurs 


Decision:  Maintain  awareness  of  aircraft 
locations  in  order  to  quickly  determine 
whether  there  is  a  conflict 

Decision:  Determine  location  of  aircraft 
within  3-dimensional  space 


Cue:  Absence  of  tracks  where  there  should 
be  tracks  (seeing  what  is  not  there) 


Decision:  Determine  whether  the  sensors 
can  be  trusted 

Decision:  Determine  malfunction 
Cues/factors:  Uses  vast  technical 
knowledge  and  understanding  of  the  system 
affordances  to  recognize  malfunction 


Just  then  the  Mission  Crew  Commander  Major  Warren 
came  over  to  see  what’s  wrong.  Ramsey  and  Collins 

filled  him  in  on  the  situation.  “I  need  to  know  which  Information  needs:  It  is  critical  to  know 

information  is  good.  Otherwise  we  are  as  good  as  blind  which  information  is  accurate 

in  this  area,”  stated  Collins.  Ramsey  explained  that  there 

was  also  a  problem  with  one  of  the  laser  sensors.  Both 

problems  required  immediate  attention. 
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Warren  considered  the  situation.  Without  those  displays 
the  Airborne  Laser  is  at  risk  and  we  may  not  be  able  to 
destroy  any  missiles.  But  I  need  the  laser  sensors  to 
make  sure  the  whole  place  doesn’t  blow  up.  Major 
Warren  then  realized  an  important  piece  of  information  - 
there  were  backup  systems  programmed  to  sound  a 
warning  if  pressure  or  heat  rose  above  certain  levels. 
Warren  ordered  Ramsey  to  divert  his  attention  to  the 
display  problem.  “I’m  giving  you  5  minutes  to  see  what 
you  can  do  with  the  link.  We’ll  have  to  rely  on  the 
backup  sensors  while  you  fix  the  link.” 

Ramsey  returned  to  his  computer  console  and  halted  the 
sensor  diagnostics  so  he  could  use  all  resources  on  the 
link  issue.  Ramsey  began  to  work  and  Warren  looked  at 
his  watch.  If  we  can’t  bring  up  the  link,  we  cannot 
continue  our  mission.  As  Warren  prepared  to  radio 
Airborne  Command  and  Control  and  update  them  on 
their  situation,  Ramsey  reported  that  he  had  isolated  the 
problem  and  would  have  the  system  running  in  a  matter 
of  moments. 

That’s  better  thought  Collins  as  new  tracks  flashed  on 
his  screen.  The  displays  were  now  consistent.  The 
updated  air  picture  was  not  surprising.  Air  superiority 
was  demonstrated  early  in  the  week.  Since  then  there 
had  been  only  sporadic  air  activity  by  the  enemy.  The 
enemy  activity  that  occurred  was  mostly  harassment  of 
friendlies  near  the  border.  That  had  been  true  today  as 
well.  In  fact,  the  only  activity  in  the  last  few  hours  was 
the  systems  going  down. 

Thirty  minutes  later  that  picture  changed.  The  Air 
Surveillance  Officer,  Major  Krasneiwski  (Kraz)  was  the 
first  to  notice  anything.  “Four  MiGs  rapidly  approaching 
the  border  from  the  east.  It  looks  like  we  might  see  some 
action,”  reported  Kraz.  The  other  crew  members  sat  up 
straighter  in  their  seats.  They  were  all  thinking  the  same 
thing.  Enemy  air  attacks  were  often  accompanied  by 
missile  launches.  “How  long  till  we  reach  the  end  of  our 
orbit?”  asked  Warren.  A  quick  look  at  the  display 
showed  that  they  were  not  in  optimal  position  to  defend 
against  a  launch.  “Our  current  orbit  won’t  bring  us 
around  into  position  for  about  15  minutes  or  so,”  replied 
Kraz.  That’s  not  soon  enoug/z  thought  Warren.  He 
requested  the  pilot  to  begin  turning  immediately  to 
provide  maximum  engageability.  The  launch  is  likely  to 
come  from  two  areas,  based  on  what  we’ve  seen  in  the 
last  few  days.  If  we  don’t  turn  now,  we  won’t  be  able  to 
cover  them  both. 


Decision:  Prioritize  to  determine  which 
system  to  fix  first 

Perform  mental  simulation  to  understand 
implications 

Cue/factor:  Backup  warning  sensor  exit 
(needs  mental  model  of  the  system  to 
determine  this) 

Teamwork:  Establish  priorities  and 
deadlines 

Decision:  Mission  go/no-go 
Decision:  Whether  to  retrograde  and  where 
Cue:  Status  of  the  systems,  are  the  laser 
functions  affected?  Will  they  be  able  to 
respond  to  a  launch?  Will  the  ABL  be  able 
to  perform  self  protection? 


Cues/factors  to  maintaining  SA:  Radar 
tracks,  typical  enemy  actions  and  trends, 
general  air  picture 

Need  to  keep  engaged  in  the  task,  even  . 
during  inactive  times 


Expectancies  violated  —  enemy  is  not 
following  their  normal  pattern 

Decision:  Anticipate  launches 

Cue:  Enemy  air  attacks  often  precede 

launches 

Decision:  Time  to  reach  end  of  orbit 
Judgment:  Is  ABL  in  optimal  position  to 
engage  missiles? 

Decision:  When  and  where  to  turn  - 
dependent  on  current  speed  and  direction, 
anticipated  targets 
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A  familiar  “BEEP”  confirmed  their  expectations  and  a 
flutter  of  activity  began.  Collins  looked  at  his  screen  to 
find  the  location  of  the  launch.  Fortunately,  the  Airborne 
Laser  was  aligned  in  the  correct  orientation  to  cover  the 
missiles  while  completing  the  turn.  A  quick  scan  of  the 
laser  sensors  showed  that  the  system  was  ready  to  fire. 


Collins  remembered  that  one  of  the  sensors  was 
malfunctioning.  “Ramsey,  did  you  ever  get  that  sensor 
fixed?”  he  asked  the  technician.  “The  laser  is  ready  to 
fire,”  responded  Ramsey.  Meanwhile,  the  system  (in 
automatic  mode)  locked  onto  a  track  and  prepared  to  fire 
as  soon  as  the  missile  came  into  range.  Collins 
continued  to  monitor  the  situation  and  saw  no  reason  to 
stop  the  shot. 


Notice  launch  and  locate  the  missile  track 
amidst  other  tracks 

Decision:  Are  potential  missile  sites 
covered? 

Decision:  Is  the  system  ready  to  fire? 
Cues:  Amount  of  laser  fuel,  sensors 
indicating  status  of  laser  components 

Decision:  Determine  accuracy  of  sensors 
Decision:  Whether  to  stop  the  laser  from 
firing 

Cues/factors:  Presence  of  higher-priority 
missiles,  whether  the  laser  is  functioning 
correctly,  conflict  with  aircraft  (however, 
this  is  handled  by  the  system  when  in 
automatic  mode) 


Suddenly  the  Weapons  System  Officer  noticed  another 
missile  launch  on  the  screen.  “Another  two  missiles 
launched  at  the  south  site,”  he  announced.  Warren  thinks 
the  south  site  is  the  Colonel’s  priority.  I  wonder  if  we 
have  time  to  get  them  all.  “How  long  until  we  fire  at  the 
first  missile?”  Warren  asked  Collins.  “The  missile 
should  be  within  range  in  a  few  seconds,”  he  responded. 
Warren  decided  to  continue  the  attack  on  the  first  missile 
since  they  were  so  close  to  firing.  Seconds  later  the  laser 
fired,  taking  out  the  first  missile,  and  the  system  locked 
onto  the  next  missile.  Although  the  system  had  hooked 
onto  the  target  and  the  missile  was  within  range,  the  laser 
still  did  not  fire. 

Collins  noticed  that  the  missile  was  coming  towards  the 
Airborne  Laser  at  an  angle  that  made  it  almost 
impossible  to  lock  onto.  I  don ’t  think  there’s  enough 
time  to  lock  onto  the  next  missile,  my  best  bet  is  to  stay 
with  this  one  and  hope  it  levels  out  in  time.  At  the  last 
moment  the  missile  changed  trajectory  and  the  laser  shot 
it  down.  Meanwhile,  the  third  missile  had  burnt  out. 
They  were  able  to  shoot  down  two  of  the  three  missiles. 
We  should’ve  had  that  one  thought  Collins  as  he 
estimated  the  projected  impact  point  of  the  leaker. 
Warren  reported  the  results  and  projected  impact  point  to 
Airborne  Command  and  Control  and  the  rest  of  the  crew 
resumed  preparations  for  the  next  launch. 


Decision:  Abort  firing  on  the  first  missile  to 
try  to  shoot  down  the  other  two  missiles? 
Cues:  Prioritization  (type  of  missile,  angle 
of  the  missiles,  launch  site,  projected 
impact  point,  do  any  of  the  missiles  carry 
weapons  of  mass  destruction,  battle 
management  mode,  commander’s  intent), 
time  (is  there  enough  time  to  shoot  all  the 
missiles  before  they  bum  out),  chances  of 
shooting  down  all  missiles  (what  missile  do 
you  have  the  best  chance  of  shooting  down) 


Decision:  Determine  why  the  laser  is  not 
firing 

Cues:  The  missile  is  at  a  bad  angle  and  the 
system  cannot  lock  onto  it 
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The  story  demonstrates  the  usefulness  of  a  scenario  as  a  tool  to  illustrate  the  cognitive  requirements  in 
context,  and  provides  a  platform  for  understanding  the  implications  of  different  cognitive  and  team 
requirements.  For  example,  variables  such  as  staff  size,  the  assignment  of  responsibilities  (including 
decisions),  and  coordination  could  be  manipulated  to  determine  the  effect  on  a  particular  mission.  The 
use  of  scenarios  is  the  basis  of  a  tool  proposed  for  follow-on  development. 

Human-Computer  Interaction  Recommendations 

These  suggestions  are  based  on  the  decisions  elicited  for  the  Battle  Management  Crew.  They  are  briefly 
described  in  the  “Additional  CTA  Data”  column  of  the  Cognitive  Challenges  tables  in  Appendix  A. 
These  tables  make  explicit  the  decisions  on  which  the  suggestions  are  based.  Some  of  these 
recommendations  are  to  add  functionality  or  concepts  to  the  displays.  These  recommendation  are 
designed  to  alleviate  some  of  the  cognitive  challenges  and  make  the  tasks  and  decisions  more 
manageable.  Other  recommendations  are  based  on  our  observations  of  how  the  operators  used  the 
technology  during  the  JEFX-99  simulation.  These  are  suggestions  for  modifications  of  the  system  and 
screen  displays.  We  recognize  one  main  purpose  of  the  ABL  participation  in  the  simulation  was  to  test 
these  displays  and  that  they  may  have  already  been  modified.  Nevertheless,  we  believe  these 
recommendations  will  be  helpful  in  making  the  displays  more  decision-focused.  A  third  type  of 
recommendation  is  to  keep  display  concepts  used  during  JEFX-99.  These  display  concepts  were 
observed  to  be  helpful  in  accomplishing  some  of  the  tough  decisions.  The  numbers  correspond  to 
numbers  in  the  tables  in  Appendix  A. 

[1.1]  Include  designators  for  tracks.  During  the  simulation,  poor  designators  caused  operators  to  refer  to 
“that  guy  over  there  in  the  north  sector.”  The  more  specific  the  operators  can  be  in  their  communication, 
the  better  they  will  be  able  keep  situation  awareness  of  the  enemy. 

[1.7]  Put  circles  around  the  Surface  to  Air  Missile  (SAM)  sites  that  indicate  the  range  missile 
effectiveness.  This  will  give  operators  a  visual  representation  of  the  areas  that  should  be  avoided.  It  will 
help  operators  gauge  the  threat  to  the  ABL  by  making  at  least  one  potential  threat  readily  apparent. 

[2.2]  Operators  need  to  be  able  to  filter  and  sort  track  information  quickly.  At  a  basic  level,  operators 
need  to  differentiate  between  tracks  and  be  able  to  turn  them  on  and  off.  In  order  to  maintain  the  big 
picture,  the  operators  should  be  able  to  access  every  kind  of  track  -  surface,  air,  ground,  and  subsurface. 
Operators  also  need  to  be  able  to  choose  which  tracks  to  leave  on.  In  order  to  do  this  quickly,  the  track 
selection  options  should  not  be  deeply  embedded  within  menus.  One  option  is  to  have  categories  from 
which  to  select,  similar  to  the  AWACS  category  select  switch. 
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[2.5]  Allow  operators  to  put  their  own  designators  on  tracks.  For  example,  if  an  operator  becomes 
suspicious  of  an  enemy  air  track  that  is  headed  in  the  direction  of  the  ABL,  the  operator  could  turn  that 
track  a  different  color  or  put  a  circle  around  it  to  easily  and  quickly  locate  the  track  on  the  screen.  This 
will  help  operators  keep  track  of  potential  problems. 

[2.7]  Allow  operators  to  switch  configurations  between  the  crew  without  re-logging  on  as  a  new  user 
(which  can  take  precious  time).  For  example,  if  one  crew  member  is  temporarily  filling  in  for  another, 
that  crew  member  could  temporarily  switch  configurations  to  see  either  the  big  picture  (zoomed  out  with 
more  tracks  active)  or  a  more  narrow  picture  (zoomed  in  with  only  the  air  tracks  active)  depending  on  the 
situation. 

[2.9]  Have  the  high  value  assets  (HVA)  stand  out  on  the  screen  so  they  are  easily  recognized.  The  ABL 
is  dependent  on  one  HVA,  the  AW  ACS,  for  much  of  its  information;  it  is  important  that  the  operators  be 
able  to  quickly  locate  HVAs.  During  the  simulation  one  of  the  operators  kept  asking  for  track  history. 
When  asked  why  he  wanted  the  history,  he  replied  that  he  wanted  to  know  where  the  AW  ACS  was  and 
track  history  would  tell  him  this.  The  operator  didn’t  need  track  history,  he  merely  needed  to  know  the 
location  of  HVAs.  The  standard  designator  of  a  HVA  could  be  a  green  circle  around  the  track. 

[2.18]  Create  an  automated  Air  Tasking  Order  (ATO)  that  operators  could  access  from  their  displays. 

For  each  mission  print  out  a  “cheat  sheet”  that  includes  the  call  signs,  modes,  and  codes.  This  is  a  lot  of 
information  to  keep  in  memory  and  call  upon  when  needed.  These  aids  would  assure  that  the  information 
is  there  when  the  operators  need  it.  In  addition,  provide  a  desk  or  writing  area  at  each  workstation  so 
operators  can  store  this  information  and  make  any  notes  that  are  necessary. 

[3.3]  If  the  system  ever  contains  an  automatic  function  that  provides  flight  directions  (changes  to  orbit, 
changes  to  position  in  orbit)  to  the  Flight  Crew,  this  information  also  needs  to  be  provided  to  the  Battle 
Management  Crew.  The  Battle  Management  Crew  will  use  this  information  to  determine  if  they  are 
ready  to  respond  to  missile  launches,  and  to  monitor  their  location  in  the  orbit. 

[3.5]  Have  the  system  record  information  on  individual  missile  launches  such  as  time  of  launch,  location, 
track  number,  actions  taken,  and  results.  Allow  the  user  to  access  either  information  on  individual 
launches  or  a  summary  of  recent  launches.  This  will  help  the  user  detect  patterns  and  determine  trends  of 
launches. 

[3.6]  Include  a  function  in  which  the  system  takes  the  information  generated  in  [3.5]  and  predicts  the 
location  of  future  launches.  It  is  important,  not  just  to  provide  the  predictive  data,  but  to  allow  the  user  to 
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access  the  raw  data  so  the  user  can  use  his/her  experience  to  look  for  trends  and  patterns.  This 
information  can  help  the  ABL  crew  determine  the  optimal  orbit  placement. 

[3.8]  Include  a  function  that  can  calculate  the  range  between  objects.  For  example,  if  the  user  selects  the 
ABL  track  and  then  selects  another  track,  the  system  will  provide  the  distance  between  the  two  objects. 

[5.2]  Add  a  function  to  the  system  that  would  use  information  such  as  anticipated  launch  sites,  current 
orbit,  current  threats  to  ABL,  and  priorities  to  calculate  and  recommend  the  optimal  orbit. 

[5.3]  There  are  many  reasons  that  the  system  will  not  fire  in  automatic  mode.  Inability  to  acquire  the 
target,  the  missile  is  no  longer  in  the  boost  phase,  a  conflict  with  another  aircraft  or  satellite,  or  the 
presence  of  a  higher  priority  target  are  a  few  examples  of  why  the  system  may  not  fire.  When  the  system 
cannot  fire,  it  is  important  that  the  operator  know  the  reason.  The  system  could  display  a  brief  message 
such  as  “Missile  out  of  boost  phase”  or  “Conflict.”  It  is  very  important  that  the  operator  understand  the 
system’s  reasoning.  If  not,  the  operator  could  switch  to  manual  mode  and  proceed  with  the  shot  anyway. 

It  there  is  a  conflict  and  the  operator  fires  in  manual  mode,  the  results  could  be  disastrous.  Another  way 
to  avoid  firing  when  a  conflict  exists  is  to  display  the  word,  “Conflict”  and  have  the  conflict  blink  on  the 
screen  so  the  operator  does  not  fire  until  the  conflict  is  resolved. 

[5.6]  Allow  the  user  to  easily  determine  how  long  it  will  be  until  the  end  of  the  orbit  is  reached.  If  the 
user  could  simply  click  on  the  end  of  the  ABL  orbit  to  determine  the  range  and  the  estimated  time  until 
arrival,  this  information  could  be  used  to  determine  if  the  aircraft  was  optimally  placed  to  respond  to 
missile  launches.  Many  times  during  JEFX-99  we  heard  the  operators  ask  the  pilot  for  this  information 
and  adjust  the  aircraft  position  based  on  that  information.  The  system  should  specify  whether  the  results 
are  in  miles  or  kilometers. 

[6.1]  Currently  in  manual  mode  the  operator  must  perform  deconfliction.  This  is  a  challenging  task 
because  the  operator  must  combine  information  from  one  display  that  shows  altitude  and  another  display 
that  shows  the  bird’s  eye  perspective  of  the  battlefield.  There  is  not  a  one-to-one  correlation  between  the 
displays  and  some  transformations  must  be  performed  to  understand  the  three-dimensional  space  and 
determine  if  there  are  conflicts.  One  solution  is  to  show  the  two  displays  from  the  same  perspective. 
Currently  the  altitude  display  is  shown  from  the  perspective  of  the  ABL  so  as  the  ABL  turns,  the  picture 
changes.  However,  the  bird’s  eye  view  display  is  shown  independent  of  ABL  movement.  In  other  words, 
on  the  altitude  display  the  ABL  stays  in  place  and  the  other  tracks  move  around  it  and  in  the  bird’s  eye 
view  display  the  ABL  moves  as  well  as  the  other  tracks.  Another  solution  is  to  incorporate  a 
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deconfliction  warning  in  manual  mode.  The  operator  could  still  decide  to  take  the  shot  but  a  warning 
message  would  appear  and  the  conflict  would  blink  on  the  screen. 

[6.7  ]  When  a  missile  is  launched,  dim  all  the  other  tracks  so  the  missile  is  easily  noticed.  This  way  the 
other  tracks  can  still  be  seen  but  the  missile  track  will  stand  out.  During  the  simulation,  the  operators  had 
trouble  finding  the  missiles  on  their  screen  during  a  launch.  The  missile  would  initially  blink  but  if  the 
operator  missed  that  signal,  the  missile  could  become  hidden  in  the  other  screen  clutter.  One  operator 
developed  a  workaround  to  compensate  for  this.  Every  time  the  audio  alarm  signaled  a  launch  the 
operator  would  clear  all  tracks  to  easily  isolate  the  missile. 

[7.13]  When  a  missile  is  launched  the  operator  may  want  to  zoom  out  or  zoom  in  to  get  a  better  picture. 
These  functions  should  be  easily  accessed  by  buttons  on  the  screen  without  having  to  access  detailed 
menus.  Another  idea  is  to  have  a  “back”  zoom  button  that  reverts  to  the  last  magnification.  This  would 
allow  users  to  toggle  between  two  relevant  magnifications. 

[7.15]  When  a  missile  is  splashed,  display  the  track  number  on  the  screen  for  a  longer  period  of  time  and 
allow  this  information  to  be  accessed  later.  During  the  simulation  the  track  number  would  sometimes 
disappear  from  the  screen  before  operators  could  read  it. 

TIDE  Analysis  of  Airborne  Laser  Crews 

This  section  describes  the  results  of  the  TIDE  analysis  on  ABL.  One  purpose  of  the  analysis  was  to 
determine  optimal  crew  configuration  including  crew  size  and  assignment  of  roles  and  responsibilities. 
The  other  purpose  was  to  ascertain  the  usefulness  of  TIDE  for  the  project  manager  tool. 

The  tasks  were  allocated  to  the  five  crew  members  (Mission  Crew  Commander  or  MCC,  Air  Surveillance 
Officer  or  ASO,  Weapon  System  Officer  or  WSO,  Technician,  Pilot)  based  on  the  current  descriptions  of 
the  ABL  concepts  of  operations.  Relative  workload  for  each  crew  member  was  calculated  across  time  for 
the  mission  functions.  Figure  6  presents  the  workload  profiles  for  each  of  the  crew  members  and  the 
pilot.  Note,  this  measurement  of  workload  was  relative  due  to  the  lack  of  detailed  information  about  the 
tasks. 

As  you  can  see  in  this  figure,  the  current  task/personnel  assignments  result  in  the  ASO  carrying  a  much 
larger  workload  than  the  rest  of  the  crew  members  across  all  overall  tasks,  particularly  in  the  missile 
elimination  tasks.  The  MCC  has  peaks  of  relatively  heavy  work  load  during  self  defense  tasks,  and  the 
WSO  and  technician  remain  relatively  low  in  workload  throughout  the  mission.  As  expected,  the  WSO’s 
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workload  increases  during  the  missile  elimination  task,  and  the  technician’s  workload  rose  in  response  to 
system  and  laser  failures. 

The  current  crew/task  assignment  was  then  examined  to  determine  where  improvements  could  be  made  to 
more  evenly  distribute  the  workload.  The  primary  objective  was  to  lower  the  ASO’s  relative  workload 
level  and  level  the  workload  distribution  across  the  entire  team.  Much  improvement  was  achieved  by 
assigning  exclusively  to  the  MCC  many  of  the  tasks  for  which  the  MCC  and  ASO  had  previously  shared 
joint  responsibility.  A  second  major  improvement  resulted  from  the  allocation  of  the  ASO’s  deconfliction 
tasks  to  the  WSO.  As  with  the  MCC’s  assignments,  most  of  the  tasks  assigned  to  the  WSO  had 
previously  been  the  joint  responsibility  of  the  ASO  and  the  WSO.  After  reallocating  the  tasks  within  the 
team,  workload  of  all  crew  members  was  recalculated,  as  shown  in  Figure  7.  As  can  be  seen,  the 
reallocations  resulted  in  a  more  evenly  distributed  workload  among  crew  members. 

The  research  team  decided  to  initiate  a  second  task  allocation  process  to  explore  the  possibility  of 
reducing  the  number  of  organization  members  (three-person  crew  plus  pilot).  Based  on  a  workload 
analysis,  the  tasks  of  the  technician  were  to  be  reallocated  to  the  other  crew  positions.  The  WSO  was 
assigned  several  of  these  tasks,  as  they  overlapped  with  many  of  his  or  her  duties.  The  initial  iteration  of 
the  reduced  staff  design  yielded  a  relatively  overloaded  WSO.  In  the  successive  iterations,  we  aimed  to 
optimize  the  workload  distribution  of  the  reduced  staff  crew.  In  a  second  iteration,  the  pilot  was  assigned 
tasks  from  the  WSO  (Monitor  cap  location)  and  tasks  from  the  ASO  (Tasks  related  to  self  defense 
monitoring).  The  result  of  these  reallocations  was  an  improvement  in  both  the  WSO’s  and  ASO’ s 
workloads,  but  a  pilot  who  was  at  times  relatively  overloaded  (particularly  when  the  ABL  was  under 
attack).  In  the  third  iteration,  the  pilot’s  task  load  was  reduced  and  focused  primarily  on  the  WSO’s 
former  duty  of  “Monitor  CAP  location.”  The  result  of  this  was  a  much  more  evenly  distributed  workload 
throughout  the  aircraft.  Figure  8  presents  the  workload  profiles  for  the  final  reduced  staff  organization. 

In  our  Phase  I SBIR  work,  we  demonstrated  that  the  TIDE  team-design  process  was  able  to  provide 
insight  regarding  where  and  how  members  of  the  ABL  crew  are  likely  to  be  overloaded  under  the  current 
team  configuration.  We  also  showed  how  the  TIDE  process  can  be  used  to  generate  a  new  team  design 
that  balances  the  workload  more  evenly  across  the  five-person  ABL  crew.  Finally,  we  explored  the 
possibility  of  reducing  the  ABL  team  size  from  five  to  four,  and  concluded  that  this  would  result  in 
possibly  unacceptable  overload  of  the  WSO  position  under  the  current  distribution  of  tasks.  When  tasks 
were  reallocated,  using  an  algorithm  to  balance  the  workload  across  crew  members,  however,  we  were 
able  to  produce  a  reduced-staff  team  design  that  accomplished  the  tasks  without  overly  burdening  any  of 
the  crew  members. 
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Figure  7.  Workload  profiles  of  crew  and  pilot  following  task  reallocation. 
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Figure  8.  Workload  profiles  of  reduced  crew  organization:  Final  iteration. 


CONCLUSIONS 


As  previously  stated,  cognitive  engineering  does  not  adequately  address  the  decision  making  or  team 
aspects  of  users  in  the  development  of  new  systems.  For  this  reason,  the  objectives  of  this  effort  were  to: 

•  target  Program  Managers  who  could  use  these  cognitive  requirements  early  in  the  design  phase, 
rather  than  system  engineers 

•  use  a  decision-centered  approach  rather  than  focusing  on  procedures 

•  study  the  emergent  properties  of  teams  rather  than  focusing  on  individuals 

We  were  able  to  successfully  demonstrate  the  effectiveness  of  the  above  concepts.  The  first  two 
objectives  were  demonstrated  within  the  context  of  ABL.  We  were  able  to  elicit  and  present  information 
about  the  critical  judgments,  decisions,  information  needs,  and  coordination  issues  involved  in  the  ABL 
mission  and  organize  these  into  major  functions  of  the  Battle  Management  Crew.  We  illustrated  several 
team  coordination  issues  in  the  decision  scenario.  A  next  step  could  be  to  distribute  the  decisions  within 
each  major  function  to  the  different  crew  members.  Taking  a  decision-centered  approach  to  task 
allocation  and  assignment  of  roles  and  functions  has  several  benefits.  It  will  be  clear  which  crew  member 
is  responsible  for  each  decision,  and  how  team  members  can  support  other  crew  members’  decision¬ 
making.  In  addition,  since  the  division  of  roles  is  based  on  an  analysis  of  the  team,  emergent  properties 
such  as  coordination  and  information  flow  will  not  be  overlooked. 

We  were  able  to  demonstrate  the  utility  of  our  third  objective  through  interviews  with  Program  Managers. 
All  four  Program  Managers  indicated  that  information  about  the  user,  including  decision  making  and 
team  aspects,  was  critical.  Program  Managers  need  this  information  early  in  the  design  process.  Ideally 
the  information  would  be  provided  in  time  to  affect  the  concept  of  operations.  If  cognitive  and  team 
requirements  can  be  incorporated  into  the  concept  of  operations,  both  time  and  money  can  be  saved.  The 
Program  Managers  expressed  disappointment  that  mature  tools  to  provide  this  information  did  not  exist. 
One  Program  Manager  emphatically  expressed  that  he  did  not  want  human  factors  “expert  opinion,”  he 
wanted  an  audit  trail  to  trace  the  origin  and  reasons  behind  recommendations  involving  the  user. 

Future  Research  Ideas 

In  addition  to  demonstrating  the  feasibility  of  our  objectives,  we  were  able  to  develop  an  approach  to 
integrating  these  ideas  into  a  comprehensive  tool.  Our  approach  would  be  to  develop  a  prototype  tool  that 
enables  Program  Managers  to  apply  cognitive  engineering  early  in  the  conceptual  phase  of  system  design. 
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Most  of  the  key  design  decisions  are  made  at  the  beginning  of  the  effort,  at  a  point  where  there  are  few 
opportunities  to  run  studies  and  collect  data  on  individual  and  team  performance.  As  a  result,  cognitive 
requirements  tend  to  be  ignored  until  relatively  late  in  the  cycle.  Our  intent  is  to  turn  that  around,  by 
providing  a  platform  for  considering  individual  and  team  cognitive  requirements  from  the  start. 

The  target  user  of  the  tool  would  be  Program  Managers  and  directors  of  engineering,  along  with  the  using 
command  sponsors  of  the  development  effort.  We  believe  that  the  tool  will  be  helpful  for  the  design 
engineers  working  on  the  system,  but  the  primary  audience  would  be  at  the  level  of  management  —  to 
conceptualize  the  dynamics  and  tradeoffs  that  involve  cognitive  tasks. 

The  prototype  tool  is  called  Cognitive  Requirements  for  Individuals  and  Teams:  Evaluations, 
Recommendations,  Integration,  and  Analysis  (CRITERIA).  The  intent  is  to  define  and  represent  the 
cognitive  criteria  for  tasks  involving  information  technology  and  command  and  control  (as  opposed  to 
physical  criteria  such  as  reach  envelopes).  The  tool  will  address  both  individuals  and  teams,  but  not 
organizations.  The  tool  will  not  contain  automated  analysis.  Our  judgment  is  that  automated  analysis  is 
not  sufficiently  mature  for  our  needs.  Instead,  CRITERIA  will  function  as  a  guide  in  collecting  and 
representing  cognitive  engineering  considerations  so  that  Program  Managers  can  consider  the 
implications  of  different  configurations. 

CRITERIA  will  include  a  graphic  display  of  team  configurations  to  show  how  alternative  configurations 
would  handle  different  types  of  situations.  This  is  the  central  aspect  of  CRITERIA.  Decision  scenarios 
will  be  developed  to  present  challenges  to  the  individuals  and  to  team  coordination,  and  CRITERIA  will 
illustrate  how  these  challenges  would  be  handled  by  teams  varying  in  size  and  qualifications.  That  is  how 
alternative  recommendations  would  be  identified  and  evaluated  early  in  the  design  cycle.  The  decision 
scenarios  and  recommendations  would  be  based  on  Cognitive  Task  Analysis  with  subject  matter  experts 
performing  analogous  types  of  work,  and  adapting  the  findings  to  fit  the  constraints  of  the  envisioned 
situation. 

CRITERIA  is  not  intended  to  provide  comprehensive  answers  about  team  configuration  questions. 

During  the  early  stages  of  concept  development,  there  are  simply  not  enough  data  and  the  goals  are  too 
ill-defined.  Instead,  CRITERIA  is  a  tool  to  assist  Program  Managers  in  thinking  about  the  issues  and 
understanding  the  tradeoffs.  We  are  drawing  on  the  work  using  decision  scenarios  (e.g.,  Schwartz,  1991) 
as  a  means  of  learning  about  dynamics,  rather  than  arriving  at  answers. 

CRITERIA  will  have  another  benefit,  which  is  to  enable  Program  Managers  to  define  a  concept  of 
operations.  The  graphic  representation  of  team  interactions,  built  around  the  results  of  Cognitive  Task 
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Analysis,  will  permit  the  designers  to  conduct  what-if  analyses,  and  to  describe  preliminary  concepts  of 
how  the  system  will  be  used  once  it  is  fielded.  The  concept  of  operations  will  contain  initial 
recommendations  for  staffing  (number,  specialty,  training  requirements)  including  command  and  control 
considerations  and  team  coordination  issues,  as  well  as  logistical  support  requirements  (if  these  are 
included  in  the  scenarios).  We  recognize  the  wide  range  of  competing  demands  placed  on  Program 
Managers,  and  we  can  see  how  easy  it  is  for  attention  to  be  focused  on  hardware  and  budgetary  concerns. 
Therefore,  we  believe  that  it  is  important  for  CRITERIA  to  assist  Program  Managers  in  tackling  essential 
parts  of  their  responsibilities  (such  as  developing  a  concept  of  operations)  to  encourage  its  use  in 
exploring  cognitive  requirements. 

Major  steps  towards  the  design  of  CRITERIA  have  been  accomplished  during  the  Phase  I SBIR  effort. 

We  have  conducted  observations  and  interviews  to  determine  the  functionality  needed.  We  have  also 
explored  the  use  of  advanced  discrete  event  simulations,  and  determined  that  the  existing  systems  would 
not  be  helpful  because  of  the  limited  amount  of  data  available  early  in  system  design.  We  determined  that 
the  focus  of  CRITERIA  needed  to  be  a  graphic  representation  showing  how  individuals  and  teams  would 
manage  different  types  of  challenges  during  operations,  so  that  Program  Managers  could  discover  for 
themselves  the  strengths  and  limitations  of  alternative  crew  configurations.  We  have  also  defined  the 
types  of  Cognitive  Task  Analysis  data  needed  to  define  the  decision  scenarios. 

We  believe  that  the  tool,  CRITERIA,  will  be  unique  in  the  type  of  capability  it  affords  Program  Managers 
to  insert  cognitive  requirements  early  in  the  design  cycle.  We  further  believe  that  CRITERIA  has  the 
potential  to  result  in  a  revolutionary  change  in  system  development.  As  the  Air  Force  draws  upon 
information  technologies  for  more  and  more  applications,  the  need  to  address  cognitive  engineering  issues 
will  grow,  and  the  value  of  CRITERIA  will  increase. 
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GLOSSARY  OF  ACRONYMS 


ABL 

Airborne  Laser 

AFSC 

Air  Force  Specialty  Codes 

ASO 

Air  Surveillance  Officer 

ATO 

Air  Tasking  Order 

AWACS 

Airborne  Warning  and  Control  System 

BMC4I 

Battle  Management  Command,  Control,  Communications,  Computers,  and 

Intelligence 

CAP 

Combat  Air  Patrol 

CRITERIA 

Cognitive  Requirements  for  Individuals  and  Teams:  Evaluations,  Recommendations, 
Integration  and  Analysis 

CTA 

Cognitive  Task  Analysis 

DCD 

Decision-Centered  Design 

HVA 

High  Value  Assets 

JEFX-99 

Joint  Expeditionary  Force  Exercise  1999 

MCC 

Mission  Crew  Commander 

SAM 

Surface  to  Air  Missile 

SBIR 

Small  Business  Innovative  Research 

TIDE 

Team  Integrated  Design  Environment 

WSO 

Weapon  System  Officer 
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APPENDIX  A 

A  DESCRIPTION  OF  THE  COGNITiVELY  CHALLENGING  ASPECTS  OF  THE  ABL 

MISSION 
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Table  A-l:  Self  Protection 


Decision/function 

CC* 

Why  challenging? 

Additional  CTA  data 

1.1  Monitor  enemy  tracks 

Med. 

There  are  multiple  tracks 
which  can  disappear  and 
reappear. 

The  enemy  will  often  be 
trying  to  use  deception  so 
operators  need  to  maintain 
enemy  perspective  to  make 
sense  of  what  they  see  on  the 
screen.  Tracks  can  carry 
weapons  that  can  be  a  threat 
from  farther  away  than  the 
original  track  itself.  Need  to 
project  ahead  and  understand 
implications  of  enemy 
actions. 

Some  classes  of  MiGs 
(MiG  25  and  31)  can  fly 
high  and  are  difficult  to 
intercept,  even  for 

F-15s. 

Display  fix  -  include 
designators  for  tracks. 

1.2  Monitor  CAP  location 

Low 

This  is  a  visual  and  auditory 
task  -  monitor  tracks  or 
radar  and  monitor 
communicates  between  CAP 
&  AWACS. 

It  may  become  more 
challenging  if  CAP  has  been 
vectored  off  somewhere 
where  they  could  become 
confused  with  other  tracks. 

Need  to  be  able  to  listen 
to  control  frequencies 
between  AWACS  and 
CAP  for  advanced  “I 
&W”  (threat  status, 
leakers,  whether  they  got 
all  the  hostiles). 

1.3  Determine  if  sensors  and 
communications  are  working 

High 

Must  notice  inconsistency  in 
data.  In  other  words,  you 
need  to  see  what  is  not  there. 

Indicators  may  not  reflect 
exactly  what  the  system  is 
actually  doing.  It  is  easy  to 
assume  that  information  is 
being  sent  and  received  if 
there  are  no  indicators  to  let 
you  know  something  is  not 
working. 

CC  stands  for  Cognitively  Challenging 
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1 

fable  A-l:  Self  Protection  (continued) 

Decision/fimction 

CC 

Why  challenging? 

Additional  CTA  data 

1.4  Determine  when  to 
retrograde  and  where 

Med. 

Need  to  project  ahead  and 
determine  when  is  it  too  late 
to  retrograde  according  to 
time  distance  affordances. 

Much  of  this  task  is 
procedural  and  specified  in 
the  Air  Tasking  Order  with 
specific  parameters  outlined. 
Difficulty  may  occur  if  there 
are  different  parameters 
depending  on  the  threat  and 
if  the  threat  has  not  been 
positively  identified.  (E.g., 
orders  to  retrograde  if  a 
certain  type  of  hostile 
aircraft  gets  within  xx  miles, 
but  provided  information  is 
insufficient  to  determine 
what  type  of  aircraft  is 
approaching.) 

1.5  Determine  mission  go/no-go 

Med. 

There  is  external  pressure  to 
continue  mission. 

Involves  early  problem 
detection. 

If  JTIDS  is  not  working, 
the  situation  may  be  a 
mission  no-go. 

Parameters  would  be 
specified  in  the  ATO. 

1.6  Determine  range  from 
threats 

Low 

If  the  Link  data  from 
AWACS,  JSTARS,  Intel, 
and  I&W  is  accurate,  this 
should  not  be  difficult. 

1.7  Gauge  threat  to  Airborne 
Laser 

Med. 

Requires  identification  of  the 
type  of  threat.  Hostile 
aircraft  can  carry  different 
kinds  of  missiles  and  it  is  the 
missile  more  than  aircraft 
type  that  can  be  the 
determining  factor. 

Need  to  anticipate  enemy 
action,  think  ahead,  and 
estimate  CAP’s  ability  to 
respond  (time,  distance 
factors). 

Display  aid  -  put  circles 
around  Surface  to  Air 
Missile  (SAM)  sites 
indicating  the  range  of 
their  missiles. 

There  is  a  need  to  know 
immediately  whether  all 
the  hostiles  were 
stopped. 

The  crew  may  know 
ahead  of  time  (from 
intelligence)  that  the 
Airborne  Laser  is  a 
target. 
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Table  A-l:  Self  Protection  (continued) 


Decision/function 

CC 

Why  challenging? 

Additional  CTA  data 

1 .8  Monitor  defensive  air  ops 
for  threats  and  potential 
engagements 

Med. 

There  are  competing 
information  sources  and 
attention  management 
issues.  Requires  attention  to 
be  focused  on  the  defensive 
air  ops  to  hear  what  they  are 
doing  and  judge  potential 
threats. 

Might  require  an 
additional 

communications  channel 
to  listen  on  defensive  air 
ops  communications. 

1 .9  Know  whether  Airborne 

Laser  is  a  target  that  day 

Low 

Requires  declarative 
knowledge. 

This  is  information  the 
enemy  would  strive  to  keep 
hidden.  Therefore,  it  may  be 
difficult  to  determine. 

May  have  to  assume  that 
Airborne  Laser  is  always 
a  target  on  any  particular 
day. 

1.10  Anticipate  CAP  refueling 

Med. 

Must  think  ahead  while 
performing  current  tasks, 
dependent  on  several  factors 
-  time,  weather,  orbit,  etc. 

Involves  communication 
with  CAP.  This  issue 
covered  the  pre-mission 
briefings. 

1.11  Decide  to  shoot  enemy 
aircraft 

Med. 

Must  be  done  in  manual 
mode  so  deconfliction  must 
be  performed  with  other 
aircraft  and  satellites. 

May  involve  visual 
identification. 

1.12  Request  to  shoot  enemy 
aircraft 

Low 
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Table  A-2.  Maintain  Plan  Flow 


Decision/function 

5T" 

CC 

Why  challenging 

Additional  CTA  data 

2.1  Monitor  enemy  tracks 

Med. 

There  are  multiple  tracks 
that  can  disappear  and 
reappear. 

The  enemy  will  often  be 
trying  to  use  deception  so 
operators  need  to  maintain 
enemy  perspective  to  make 
sense  of  what  they  see  on  the 
screen. 

Tracks  can  carry  weapons 
that  can  be  a  threat  from 
farther  away  than  the 
original  track  itself. 

Need  to  project  ahead  and 
understand  implications  of 
enemy  actions. 

2.2  Filter  and  sort  information 

High 

Need  to  make  meaning  and 
draw  conclusions  from  the 
information  (synthesize). 

If  information  is  ambiguous, 
missing,  or  incorrect,  it  can 
make  filtering  and  sorting 
difficult. 

If  too  much  information  is 
coming  in,  it  can  be  difficult 
to  identify  which 
information  is  relevant. 

Operators  need  to  filter 
the  information  that  is 
displayed.  All  data 
blocks  are  not  the  same; 
it  depends  on  the  track. 

For  example,  altitude  is 
not  necessary  for  ground 
and  surface  tracks. 

Modes  and  codes  are  not 
necessary  for  all  tracks. 

2.3  Determine  whether  sensors 
and  communications  are 
working 

High 

Must  notice  inconsistency  in 
data.  In  other  words,  you 
need  to  see  what  is  not  there. 

Indicators  may  not  reflect 
exactly  what  the  system  is 
actually  doing.  It  is  easy  to 
assume  that  information  is 
being  sent  and  received  if 
there  are  no  indicators  to  let 
you  know  something  is  not 
working. 

CC  stands  for  Cognitively  Challenging 
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Table  A -2.  Ma 

Intain  Plan  Flow  (continued) 

Decision/function 

cc 

Why  challenging? 

Additional  CTA  data 

2.4  Determine  where  strikes  are 
taking  place 

Low 

Information  is  available  in 
the  Air  Tasking  Order 
(ATO)  and  on  the  screen. 

A  maneuvering  surface 
to  air  missile  (SAM) 
suggests  a  strike 
package  is  taking  place. 

2.5  Detect  problems  and 
inconsistencies  in  track  data 

High 

This  is  dependent  upon  the 
number  of  sensors  you  have 
out  there,  the  sensitivity  of 
the  sensors,  and  the  location 
of  the  sensors. 

Inconsistencies  may  not 
stand  out,  or  they  may  occur 
frequently  enough  that  the 
important  ones  do  not  stand 
out  more  than  the 
unimportant  ones. 

Need  experience  base  to 
compare  situation  and 
recognize  anomalies. 

Display  aid  -  a  tool 
could  be  built  to  track 
certain  aspects  during 
critical  salient  events. 

It  is  difficult  to  maintain 
situation  awareness  and 
not  narrow  your  focus 
during  a  critical  event. 

A  display  aid  could 
show  relevant 
information  without 
distracting  the  operator 
from  the  focus. 

2.6  Know  and  apply 
commander’s  intent 

High 

Involves  applying 
commander’s  intent  to  the 
current  situation  and 
determining  if  your  actions 
are  consistent. 

This  can  be  difficult  if  the 
commander’s  intent  is 
poorly  articulated  or  focused 
at  the  wrong  level  of 
granularity. 

2.7  Switch  configuration 
between  users 

Low 

Need  to  determine  the  level 
of  information  you  need  to 
know  (big  picture  versus 
specifics). 

This  needs  to  be  done 
quickly  without 
relogging  in  as  a  new 
user. 

If  the  display  and  its 
accompanying  switch 
actions  to  do  this  task 
are  complex  and  deeply 
embedded  in  menus,  it 
can  make  the  task  much 
more  difficult. 

2.8  Perform  spatial  and  audio 
discrimination  between  voice 
communications 

Low 

Need  to  determine  who  is 
speaking  (source  of 
information)  at  the  same 
time  you  are  listening  for 
relevant  information. 
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Table  A-2.  Maintain  Plan  Flow  (continued) 


Decision/function 

CC 

Why  challenging? 

Additional  CTA  data 

2.9  Monitor  location  of  high 
value  assets  (HVAs) 

Med. 

Can  be  difficult  if,  for 
security  reasons,  the 
locations  of  the  HVAs  are 
not  widely  disseminated. 

Some  individuals  on 

Airborne  Laser  may  not 
have  high  enough  clearance 
to  be  entitled  to  this 
information. 

Note:  “standard”  HVAs 
shouldn’t  pose  too  much  of  a 
problem  since  they  should 
be  readily  visible  on  the 
displays  (e.g.,  AWACS, 
JSTARS,  RJ,  EA-6B,  etc.). 

Display  aid  -  have  the 

HVAs  stand  out  by  placing 
a  circle  around  them  or 
using  a  different  color. 

2.10  Monitor  track  number 
changes 

Med. 

Need  to  integrate 
information  from  multiple 
sources. 

If  the  track  number  changed, 
the  operator  may  need  to 
unlearn  the  old  number  as 
well  as  learn  the  new 
number. 

It  is  possible  that  tracks 
may  be  picked  up  by 
multiple  sensors,  especially 
in  joint  operations  (e.g.,  by 
a  US  AWACS,  a  NATO 
AWACS,  an  E-2C,  an 

AEGIS  cruiser).  Given  the 
possibility  for  multiple 
sensors  to  acquire  the  track, 
there  is  a  need  for  all  of 
these  linked  data  sources  to 
be  “correlated.”  There  is  a 
greater  chance  that  tracks 
can  be  given  multiple  track 
numbers  until  the  system 
sorts  out  what’s  actually  out 
there.  As  a  result,  there 
could  be  situations  where  a 
track  number  could  be 
changed. 

One  difficulty  with  track 
number  changes  is  that 
operators  often  type  in  the 
track  number  of  interest 
(once  they  ‘learn’  it)  and  a 
change  in  numbers  can  have 
them  inadvertently  looking 
at  the  wrong  track. 
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Table  A-2.  Maintain  Plan  Flow  (continued) 


Decision/function 

CC 

Why  challenging? 

Additional  CTA  data 

2.1 1  Monitor  defensive  air  ops 
for  threats  and  potential 
engagements 

Med. 

There  are  competing 
information  sources  and 
attention  management 
issues.  Requires  attention  to 
be  focused  on  the  defensive 
air  ops  to  hear  what  they  are 
doing  and  judge  potential 
threats. 

Might  require  an 
additional 

communications  channel 
to  listen  on  defensive  air 
ops  communications. 

2.12  Determine  which  links  to 
leave  on 

Med. 

One  challenge  is  to 
determine  what  information 
is  needed  and  anticipate 
future  information  needs,  to 
maintain  SA. 

Another  challenge  is  to 
figure  out  which  data  source 
is  the  accurate  one.  Almost 
every  source  believes  thev 
are  sending  accurate 
information. 

2.13  Input  and  output  the  order 
of  battle 

Low 

2.14  Monitor  voice  product  net 
for  C2  information 

Med. 

Requires  attention  to 
monitor. 

Criticality,  importance,  and 
implications  of  information 
may  not  be  obvious  so  it 
may  require  additional 
analyses  to  interpret. 

2. 1 5  Keep  the  team  engaged 
and  alert  to  the  possibility  of 
missile  launches 

Med. 

Crew  will  need  to  keep 
vigilance  and  prevent 
complacency  or  boredom. 

The  crew  needs  to  maintain 
situation  awareness  in  order 
to  act  immediately  when  a 
launch  occurs. 

One  estimate  is  that  95% 
of  the  time  the  crew  will 
be  inactive. 

2.16  Edit  automatic 
prioritization  matrix 

Low 

2.17  Update  pilot  (enemy 
location,  status) 

Med. 
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Table  A-2.  Maintain  Plan  Flow  (continued) 


Decision/function 

CC 

Why  challenging? 

Additional  CTA  data 

2.18  Reassign  tasks/orchestrate 
priorities 

High 

This  is  a  resource  allocation 
issue.  It  is  difficult  to 
determine  priorities  when 
everything  is  important. 

This  must  be  done  on  a  minute- 
by-minute  basis. 

One  strategy  is  to  ask 
how  long  a  task  will  take 
and  determine  if  there  is 
something  that  should  be 
done  in  the  meantime. 

2.19  Know  the  players  call  sign, 
modes,  and  codes 

Med. 

This  is  a  lot  of  information  to 
commit  to  memory  and  call 
upon  when  needed. 

Display  aid  -  an 
automated  ATO  or  cheat 
sheet. 

2.20  Anticipate  refueling 

Low 

Involves  projecting  into  the 
future  to  determine  need. 
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Table  A-3.  Aircraft  Placement 


Decision/function 

*  I 

CC 

Why  challenging 

Additional  CTA  data 

3.1  Determine  placement  of 
orbit 

Med. 

This  is  a  tradeoff  because 
the  Airborne  Laser  needs  to 
be  close  enough  to  pick  up 
launches  but  far  enough 
away  so  as  to  not  be 
constantly  at  risk  by  SAMs, 
hostile  aircraft,  or  other 
enemy  threats. 

3.2  Determine  type  of  orbit 
(circle,  figure  eight) 

Med. 

Need  to  develop  new 
strategies  to  maximize  the 
time  pointed  towards  threat 
areas  and  to  place  the 
Airborne  Laser  close 
enough  to  threats  that  they 
can  acquire  and  shoot 
missiles  down  before  they 
bum  out. 

Placement  of  orbit  is 
driven  by:  terrain 
(mountains,  tree 
canopy),  assets,  threat, 
battle  management 
mode,  whether  the 
Airborne  Laser  is  a 
target,  status  of  weapon 
system,  etc. 

Will  probably  need  to 
consider  some  additional 
geometrical  orbit 
configurations. 

Initial  orbit 

configurations  seem  to 
be  based  on  those  flown 
by  AWACS  &  JSTARS 
and  these  may  not  be 
good  base  models  for 
determining  the 

Airborne  Laser’s  ideal 
orbit  for  a  particular 
mission. 

3.3  Monitor  position  in  orbit 
&  range  of  fire 

High 

Involves  anticipating 
launches  (thinking  ahead) 
and  determining  where  the 
aircraft  will  be  during 
critical  events.  In  addition, 
their  priorities  (possible 
launch  areas)  need  to  be 
factored  into  orbit  design, 
and  these  can  be  conflicting 
priorities.  Involves 
tradeoffs  between  optimal 
fire  position  and  safety  of 
ABL 

Display  fix  -  any 
automatic  directions  to 
the  Battle  Management 
Crew  as  well  as  the 
flight  crew. 

CC  stands  for  Cognitively  Challenging 
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Table  A-3.  Aircraft  Placement  (continued) 


Decision/function 

CC 

Why  challenging? 

Additional  CTA  data 

3.4  Determine  how  long  it  will 
take  to  reach  the  end  of  orbit 

Med. 

This  requires  time-space 
calculations  that  are  difficult 
to  perform  mentally  with 
great  accuracy. 

3.5  Determine  trends  of  launch 
locations 

Med. 

Need  to  remember  and 
integrate  launch  locations  as 
well  as  context  of  the  launch 
(time,  situation,  threat  level) 
to  identify  trends. 

Display  aid  -  The 
system  could  summarize 
information  on  launches 
and  present  this  to  the 
user. 

When  does  something 
become  a  pattern? 

Once  you  identify  a 
“trend,”  do  you  move 
the  Airborne  Laser  and 
commit  yourselves  to 
this  trend?  How 
conservative  do  you 
want  to  be? 

3.6  Anticipate  future  launch 
sites 

Med. 

Involves  identifying  trends, 
projecting  ahead,  and 
thinking  like  the  enemy. 

Display  aid  -  The 
system  could  predict 
future  launch  sites  and 
the  likelihood  of  their 

occurrence. 

3.7  Gather  intelligence  on 
predicted  launches 

Low 

Real  time  (and  accurate) 
intelligence  may  not  be 
able  to  be  disseminated 
in  enough  time  for  it  to 
be  beneficial. 

3.8  Recommend  changes  in 
orbit  or  speed 

Med. 

Requires  consideration  of 
multiple  factors  to  determine 
if  there  will  be  an  advantage 
to  changing  the  orbit. 
(Anticipated  launch/target 
sites,  current  orbit,  threats  to 
Airborne  Laser,  priorities, 
etc.) 

The  aircraft  may  already 
be  in  a  right  bank 
although  you  need  to 
turn  left.  It  may  be 
quicker  to  keep  turning 
right  even  though  it  is 
the  longer  route. 

Display  aid  -  the  system 
could  help  calculate  this 
information  and 
recommend  orbit 
changes. 
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Table  A-3.  Aircraft  Placement  (continued) 


Decision/function 

CC 

Why  challenging? 

Additional  CTA  data 

3.9  Request  change  in  orbit 
placement 

Low 

Need  to  time  the  request  so 
the  person  who  makes  the 
decision  hears  it. 

It  may  be  difficult  to 
reach  someone  who  has 
authority  to  approve 
orbit  placement  changes. 

A  change  in  orbit 
placement  is  a 
commitment.  It  means 
moving  away  from  an 
area  that,  at  least  at  one 
time,  was  considered  a 
good  place  to  be. 

3.10  Monitor  defensive  air  ops 
to  determine  threats 

Med. 

There  are  competing 
information  sources  and 
attention  management 
issues.  Requires  focused 
attention  on  the  defensive  air 
ops  to  hear  what  they  are 
doing  and  judge  potential 
threats. 

Might  require  an 
additional 

communications  channel 
to  listen  on  defensive  air 
ops  communications. 
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Table  A -4.  Determine  System  Status 


Decision/function 

mm 

Why  challenging 

Additional  CTA  data 

4.1  Determine  quality  of  data 
(timely,  accurate) 

High 

[Processed  data  may  be 
enhanced  by  the 
computer  and  mask 
quality  so  operators  need 
to  be  able  to  notice  subtle 
cues.]  Operators  need 
experience  and  a  baseline 
to  develop  expectancies 
and  notice  violations. 

Delays  in  system 
processing  can  effect 
timelines. 

Often  there  is  not  enough 
time  to  go  back  to  the 
source  and  confirm 
information. 

The  source  can  impact 
decisions.  For  example 
if  the  source  is  ground 
radar,  there  are  certain 
things  that  they  cannot 
see  so  that  data  may  not 
be  as  credible. 

If  multiple  tracks  are 
reported,  and  they  are 
from  the  same  source,  it 
is  likely  that  multiple 
tracks  actually  exist. 
However,  if  the 
multiple  tracks  are 
reported  from  different 
sources,  they  may 
actually  be  the  same 
track  reported  twice. 

4.2  Determine  confidence  in  data 

Med. 

The  source  can  alter 
confidence  and  the  source 
may  not  be  obvious. 

The  operator  may  not 
know  what 
transformations  have 
been  performed  on  the 
data. 

Display  aid  -  The 
source  of  the 
information  could  input 
confidence  fields  (1-4 
with  1  being  slightly 
confident  and  4  being 
very  confident)  and 
update  these  as 
necessary.  The 

Airborne  Laser 
operators  could  then 
have  access  to  these 
confidence  fields. 

An  operator  needs 
experience  and 
knowledge  in  air 
theatre  to  determine 
whether  the  source  is  in 
a  position  to  “see”  the 
information. 

You  need  “smart  ears.” 
Sometimes  you  want  to 
see  data  as  well  as  hear 
it. 

CC  stands  for  Cognitively  Challenging 
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Table  A-4.  Determine  System  Status  (continued) 


Decision/function 

CC 

Why  challenging? 

Additional  CTA  data 

4.3  Assess  whether  the  systems 
are  working 

Med. 

The  system  may  mask  the 
problem.  May  need  to  pick 
up  on  subtle  cues  or  interact 
with  the  system  to  detect 
problems. 

4.4  Determine  if  the  laser  is 

High 

This  is  a  new  system  and 

Will  require  input  from 

working 

operators  will  not  have  an 
experience  base. 

technician. 

4.5  Maintain  a  mental  model  of 
system  and  affordances  (system 
capabilities  and  limitations) 

Med. 

Need  to  integrate 
information  from  multiple 
sources  to  form  a  larger 
mental  model. 

Requires  intimate 
understanding  of  the  system. 

4.6  Determine  which  data  are 
better  (more  timely,  accurate) 

Med. 

Involves  conflicting 
information. 

May  not  be  able  to 
determine  source. 

Requires  information  about 
the  “age”  of  the  data,  which 
may  be  difficult  to  obtain. 
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Table  A-5.  Readiness  to  Shoot 


J 

Decision/function 

CC* 

Why  challenging 

Additional  CTA  data 

5.1  Determine  if  the  aircraft  is 
turned  the  correct  way 

Med. 

Involves  anticipating 
launches  (thinking  ahead) 
and  determining  where  the 
aircraft  will  be  during 
critical  events.  In 
addition,  priorities 
(possible  launch  areas 
and/or  anticipated  NAIs) 
need  to  be  factored  into 
orbit  design,  and  these 
[can  be  conflicting 
priorities]  may  conflict. 

5.2  Recommend  changes  in 
orbit 

Med. 

Requires  consideration  of 
multiple  factors  to 
determine  if  there  will  be 
an  advantage  to  changing 
the  orbit.  (Anticipated 
launch/target  sites,  current 
orbit,  threats  to  Airborne 
Laser,  priorities,  etc.) 

The  aircraft  may 
already  be  in  a  right 
bank  although  you  need 
to  turn  left.  It  may  be 
quicker  to  keep  turning 
right  even  though  it  is 
the  longer  route. 

Display  aid  -  the 
system  could  help 
calculate  and 
recommend  orbit 
changes. 

5.3  Know  weapon  status/Is  the 
laser  ready  to  shoot 

Med. 

Sensors  may  be  deceiving, 
weapons  status  may  not  be 
readily  apparent. 

Operators  need  a  mental 
model  of  the  system  and 
affordances. 

The  system  may 
determine  there  is  not 
enough  time  and  decide 
not  to  fire  at  the 
missile.  If  the  system 
will  not  fire,  the 
operators  need  to  be 
informed  of  the  reason 
why. 

5.4  Know  link  status 

Med. 

Sensors  may  be  deceiving. 
Operators  need  a  mental 
model  of  the  systems  and 
affordances. 

5.5  Maintain  mental  model  of 
system  affordances  and 
capabilities 

Med. 

Operators  need  to  know 
what  the  system  is  capable 
of  under  different 
situations,  be  able  to 
match  situation  to  the 
capability. 

CC  stands  for  Cognitively  Challenging 
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Table  A-5.  Readiness  to  Shoot  (continued) 


Decision/function 

CC 

Why  challenging 

Additional  CTA  data 

5.6  Determine  range  from  objects 
(i.e.,  how  long  until  reach  the  end 
of  orbit) 

High 

This  requires  time-space 
calculations  that  are 
difficult  to  perform 
mentally  with  great 
accuracy. 

Display  fix  -  Give 
operators  the  ability  to 
click  on  a  section  of 
orbit  to  determine  time 
to  intercept. 

Be  sure  to  specify  miles 
or  kilometers. 
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Table  A-6.  Decide  Whether  to  Stop  Shot 


Decision/function 

* 

CC 

Why  challenging 

Additional  CTA  data 

6.1  Deconflict  missiles  from 
aircraft  and  satellites 

Med. 

Currently  requires 
integrating  information 
from  two  displays 
multiple  points  of  view. 

Display  aid  -  Even  in 
manual  mode,  the 
system  could  provide  a 
warning  if  a  conflict  is 
detected. 

Display  aid  -  Have 
potential  conflicts  blink 
on  the  screen. 

6.2  Maintain  mental  model  of 
satellite  locations  (orbits, 
movements,  speed,  future 
locations) 

High 

The  locations  are 
constantly  changing  and 
information  about 
location  may  not  be 
easily  obtained  because 
of  its  classified  nature. 

Requires  a  3-dimensional 
model  of  the  world  to 
know  if  there  are 
conflicts. 

6.3  Determine  type  of  missile 

High 

Operators  need  to  detect 
subtle  cues  because  many 
missiles  appear  similar. 

In  the  case  of  modified 
missiles,  there  is  no  basis 
for  comparison. 

The  highest  priority  are 
those  missiles  carrying 
weapons  of  mass 
destruction. 

How  quickly  can  the 
system  be  updated?  The 
type  of  missile  affects 
prioritization. 

Need  to  be  able  to 
distinguish  between 
SAMs  and  TBMs.  The 
Airborne  Laser’s 
mission  is  to  shoot 
down  TBMs.  The  only 
time  the  Airborne  Laser 
will  shoot  SAMs  is  in 
self  defense. 

A  history  of  missile 
origins  as  well  as  the 
type  of  missile  would 
be  helpful.  A  library  of 
plume  signals  could  aid 
the  system  in 
identifying  the  type  of 
missile. 

CC  stands  for  Cognitively  Challenging 
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Table  A-6.  Decide  Whether  to  Stop  Shot  (continued) 


Decision/function 

CC 

Why  challenging 

Additional  CTA  data 

6.4  Determine  how  many  shots 
are  left 

High 

This  will  vary  depending 
on  time  required  for  each 
shot. 

Depends  on  angle, 
distance,  and  altitude 
for  shot.  Unless  the 
operator  knows  each  of 
these,  s/he  can  only 
estimate. 

6.5  Ensure  compliance  with  ROE 

High 

ROE  can  be  ambiguous, 
poorly  stated,  and  subject 
to  interpretation 

6.6  Know  tactics  and  strategies 

Med. 

Need  to  know  when  to 
apply  the  strategies.  This 
is  based  on  vast 
experience  and 
declarative  knowledge. 

Possible  reasons  for  no¬ 
kill:  out  of  range,  long  i 
missile,  or  bad  angle. 

6.7  Notice  launch 

Med. 

Requires  vigilance. 

Display  fix  -  Combine 
an  audible  warning 
with  a  flashing  missile 
track. 

6.8  Determine  location  of  missile 

Med. 

It  may  be  difficult  to 
discriminate  missile 
tracks  from  other  tracks, 
especially  if  they  are 
overlapping. 

6.9  Prioritize  targets 

High 

Take  multiple  things  into 
consideration,  such  as 
Battle  Management 

Mode. 

There  may  be  modified 
missiles  for  which  there 
is  incomplete  information 
and  no  basis  for 
comparison. 

If  the  enemy  has 
modified  the  missile  it 
may  be  unfamiliar  and 
not  match  anything  in 
the  database. 

The  decision  may  have 
been  made  to  shoot  and 
then  a  higher-priority 
missile  is  launched. 

There  could  be 
conflicting  inputs. 

Prioritization  is  based 
on:  special  instructions, 
ROE,  dynamic 
environment  (system 
operation,  laser  fuel, 
multiple  operations, 
station  time  left,  and 
guidance). 
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Table  A-7.  Ad  justments  during  Engagement 


Decision/function 

* 

CC 

Why  challenging 

Additional  CTA  data 

7.1  Acquire  target 

Low 

May  be  difficult  to  locate  if 
there  are  other  overlapping 
tracks  in  the  area. 

7.2  Determine  mode  (manual, 
auto,  semi) 

Med. 

Requires  knowledge  of 
capabilities  of  all  modes  to 
match  the  situational  aspects 
to  the  appropriate  mode. 

The  operator  may  want  to 
be  in  manual  if  the  system 
will  not  lock  onto  a  track  or 
to  select  another  track 
(prioritize). 

7.3  Assess  whether  the  missile  is 
at  a  good  angle  to  shoot 

Med. 

Need  to  integrate 
information  from  multiple 
displays  and  perspectives. 

If  the  missile  is  “facing”  the 
Airborne  Laser  (coming 
directly  towards  the 

Airborne  Laser),  it  is  a  non- 
optimal  shot.  It  may  be 
better  to  wait  until  the  angle 
changes. 

7.4  Determine  if  the  missile  is  at  a 
good  altitude  to  shoot 

Med. 

If  the  target  is  coming 
straight  at  the  Airborne 

Laser,  it  is  a  non-optimal 
shot.  It  may  be  better  to 
wait  until  the  missile 
trajectory  changes. 

7.5  Determine  why  the  system 
won’t  engage 

High 

Need  detailed  knowledge  of 
the  system  and  no-shoot 
parameters. 

This  information  may  not  be 
apparent.  If  not  obvious, 
there  is  no  way  to  find  out. 

Display  aid  -  the  system 
could  display  “no  fire”  if  a 
satellite  is  in  the  way. 

7.6  Track  missile  progress/status 

Low 

7.7  Gauge  optical  turbulence 

High 

Must  be  able  to  understand 
implications  of  varying 
degrees  of  turbulence. 

The  atmospheric  model  of 
the  day  could  be  success  or 
failure  oriented. 

May  require  a  sophisticated 
display  to  “get  a  sense  of’ 
optical  turbulence. 

7.8  Judge  time  to  hook,  lock, 
engage 

Med. 

Space,  time,  movement,  and 
speed  integration  required. 

7.9  Determine  how  long  to  wait  to 
shoot 

High 

Involves  risk  assessment 
and  tradeoffs. 

The  closer  the  missile,  the 
higher  the  kill  rate  and  the 
shorter  the  shot.  However 
there  is  a  higher  risk  of  the 
missile  burning  out  or 
dropping  too  low. 

CC  stands  for  Cognitively  Challenging 
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Table  A-7.  Adjustments  during  Engagement  (continued) 


Decision/function 

CC 

Why  challenging 

Additional  CTA  data 

7.10  Determine  how  long  to 
engage  target 

Med. 

There  are  a  limited  number 
of  shots  per  mission, 
completing  goals,  and 
resource  allocation  issues. 

The  longer  the  engagement, 
the  more  fuel  is  used. 

7.1 1  Isolate  missiles 

Med. 

Need  to  quickly  manipulate 
the  screen  and  zoom  in  to 
select  one  missile  from  a 
group  of  missiles. 

Display  fix  -  Provide  a 
back  button  to  easily  zoom 
in  and  out  between  different 
magnifications. 

7.12  Manipulate  screen  interface 

Low 

7.13  Report  results 

Low 

Display  fix  -  display 
information  on  the  screen 
for  a  longer  period  of  time. 
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Table  A-8.  Deal  with  Emergency  Situations 


Decision/function 

ism 

Why  challenging 

Additional  CTA  data 

8.1  Decide  whether  to  perform 
emergency  landing 

High 

Must  determine:  is  it  safe, 
other  alternatives,  best  location 
to  land,  and  implications. 

May  require  mental  simulation 
of  potential  events. 

8.2  Decide  whether  to  perform 
fuel/chemical  dump 

High 

Must  determine:  is  it  safe, 
other  alternatives,  and 
implications  of  actions. 

May  require  mental  simulation 
of  actions. 

8.3  Deal  with  chemical  leak 

High 

Must  determine:  danger  to 
crew  members,  danger  to 
aircraft,  how  should  the  leak 
be  dealt  with,  and  severity  of 
the  leak. 

8.4  Determine  danger  to  crew 

High 

Need  a  mental  model  of  the 
laser  and  laser  components  in 
order  to  project  into  the  future. 

- — 

8.5  Determine  system  status 

High 

This  may  be  deceiving. 
Operators  need  to  detect 
inconsistencies  or  violations  of 
expectations. 

8.6  Monitor  weather 

High 

Need  to  determine  what  is 
relevant,  and  how  it  will  affect 
the  mission,  laser,  and  aircraft. 

8.7  Anticipate  retrograding 

Med. 

Need  to  think  ahead,  assess 
threat  to  aircraft,  determine 
range  from  threat,  determine 
when  it  is  too  late  to 
retrograde,  and  use  this 
information  to  build  an 
understanding  or  the  world. 

8.8  Perform  visual  identification 

Low 

8.9  Realize  things  are  “not  going 
right” 

High 

Must  have  a  baseline  for 
comparison  to  notice 
inconsistencies  or  violations  of 
expectancies. 

There  are  many  things  that 
could  go  wrong  (aircraft 
mechanics,  laser,  systems, 
enemy  threats) 

CC  stands  for  Cognitively  Challenging 
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Table  A-8.  Deal  with  Emergency  Situations  (continued) 


Decision/function 

CC 

Why  challenging 

Additional  CTA  data 

8.10  Recognize  jamming 

Med. 

Need  to  be  able  to  see  what  is 
not  there  and  notice  violations 
in  expectancies. 

8.1 1  Apply  techniques  to  reduce 
degradation  from  jamming 

Med. 

Involves  troubleshooting, 
knowledge  of  the  system  and 
strategies,  and  the  ability  to 
think  like  the  enemy. 
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Table  A-9.  Keep  the  System  Working 


Decision/function 

K33H 

Why  challenging 

Additional  CTA  data 

9.1  Prioritize  which  systems  to 
troubleshoot 

High 

All  the  systems  are 
necessary.  May  need  to 
mentally  simulate  to 
determine  which  system  is 
most  critical. 

Must  maintain  a  mental 
model  of  all  the  system 
functions  to  assess  their 
effect  on  mission  objectives. 

If  AW  ACS  loses  radar  the 
Airborne  Laser  is  vulnerable 
to  attack.  If  the  link  to 

AWACS  is  down,  the 

Airborne  Laser  can  continue 
the  mission  but  is  dependent 
on  AWACS  for  self¬ 
protection. 

If  several  systems  go  down, 
which  one  do  you  fix  first? 

9.2  Switch  configurations 
between  users 

Low 

9.3  Apply  strategies 

Med. 

Requires  experience  and 
knowledge  to  understand 
strategies  and  when  to  apply 
them. 

An  abundance  of  technical 
knowledge  is  needed  to 
monitor  laser  status. 

If  the  link  goes  down,  the 
Airborne  Laser  needs  to  turn 
away  from  the  threat. 

9.4  Calibrate  and  initialize 
computers 

Low 

May  need  to  calibrate 
several  systems  at  once. 

9.5  Maintain  mental  model  of 
system  and  affordances 

Med.- 

High 

Need  to  know  what  the 
system  is  capable  of  under 
different  situations  and  be 
able  to  match  situation  to  the 
capability. 

Requires  intimate 
understanding  of  the  system. 

9.6  Maintain  link 

Med. 

Troubleshooting  involves 
noticing  subtle  cues, 
determining  what  is  wrong, 
and  applying  strategies. 

9.7  Optimize  the  IR 

Med. 

This  is  difficult  when  sensor 
suites  are  degraded. 

Resolution  and  resolve  rate  are 
lost  and  operator  will  need  to 
decide  what  to  ignore.  This 
can  be  done  by  shrinking  the 
field  of  regard. 

9.8  Request  data  update 

Low 

9.9  Determine  how  to  work  with 
degraded  systems 

Med. 

Need  to  determine  if  the 
degradation  is  too  severe  to 
continue  mission. 

May  have  to  develop 
workarounds  or  contingencies. 

CC  stands  for  Cognitively  Challenging 
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Table  A-10.  Track  Detectioii  and  Identification 


Decision/function 

CC* 

Why  challenging 

Additional  CTA  data 

10.1  Locate  tracks  by  number 

Low 

There  are  multiple  tracks 
and  operators  need  to 
account  for  track  number 
changes  (which  is  an 
unlearning  issue). 

10.2  Use  standardized  voice 
tell  formats 

Low 

Requires  declarative 
knowledge  of  voice  tell 
formats. 

10.3  Highlight  tracks 

Low 

10.4  Accept/reject  automatic 

identification 

recommendations 

Med. 

Requires  detection  of 
problems/errors  using 
subtle  cues. 

' 

10.5  Find  tracks 

Med. 

Must  know  bearing  and 
range  of  tracks. 

CC  stands  for  Cognitively  Challenging 
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